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Preface 


Because  of  funding  and  other  limitations,  the  Air  Force 
terminated  the  effort  which  this  document  describes  before 
the  effort  reached  its  logical  conclusion.  This  report  is 
not  satisfactory  but  was  published  in  the  interest  of 
capturing  and  disseminating  the  computer  security  technology 
that  was  available  when  the  effort  was  terminated. 

This  report  was  to  document  the  analysis  phase  of  a one 
year  study  to  determine  the  architecture  for  a 
communications  processor  that  could  1)  provide  effective 
internal  security  controls  by  supporting  the  implementation 
of  a software  security  kernel  2)  perform  communications 
processing  in  a variety  of  DoD  applications  and 
environments,  3)  have  performance  efficiency  comparable  to 
similar  non-secure  processors  and  4)  be  cost  competitive 
with  similar  non-secure  processors.  During  the  analysis 
phase  various  alternative  architecture  features  were 
considered.  The  above  four  criteria  guided  the  selection  of 
the  various  alternatives.  Unfortunately,  the  report 
concentrates  on  describing  the  particular  features  rather 
than  identifying  the  alternatives  and  the  analysis  that  lead 
to  the  selection  of  a particular  alternative. 

Additional,  more  specific  technical  comments  on  this 
report  are  attached  as  an  appendix. 


I 


r 


TABLE  OF  CONTENTS 

Page 

LIST  OF  ILLUSTBATICNS  5 

LIST  OF  TABLES  6 


SECTION  I INTRCEDCTICN  8 

PURPOSE  OF  STUDY  8 

BACKGROUND  8 

TECHNICAL  APPROACH  12 

DESIGN  REQUIREMENTS  14 

RECOMMENDED  DESIGN  15 

SUMMARY  AND  RECCMI1EN D ATIOfIS  17 

SECTION  II  ARCHITECTURAL  OVERVIEW  19 

I 

REFERENCE  HCNITOP  i 19 

MODELS  1 19 

ARCHITECTURAL  ALTERNATIVES  1 21 

Reference  Monitor  ! 21 

Complete  Mediation  j 21 

Isolation  j 22 

Provable  Correctness  | 22 

Efficiency  23 

MAPPING  ADDRESSES  1 23 

Map  Manipulations  1 25 

Map  Realization  - Descriptors  ■ 26 

PROTECTION  MECHANISMS  26 

Generalized  Domains  26 

Hultics  Bing  Structure  28 

SECTION  III  DESIGN  CCNCEPT  32 

INTRODUCTION  32 

LEVELS  OF  ABSTRACTION  32 

LEVELS  OF  THE  KERNEL  35 

LEVELS  4 AND  GREATER  - UNCERTIFIED  SOFTWARE  36 

LEVEL  3 - SECURITY  . ! 38 

LEVEL  2 - SEGMENTED  VIRTUAL  MEMORY  39 

LEVEL  1 - SEQUENTIAL  PROCESSES  41 

LEVEL  0 - THE  HAPEWAPE  44 


1 


i 

i 

\ 


TABLE  OF  CONTENTS  (concluded) 

i 

INTSODaCTICN  46  i 

THE  SPM  50  i 

SUPPORT  OP  THE  PROCESS  ABSTRACTION  50 

sdppcpt’op  the  augmented  main  memory  ABSTHACTICN  53 

SUPPORT  FOB  ACCESS  CONTROL  AND  RINGS  58 

performance  CONSIDERATIONS  '58 

SUPPORT  0?  MULTIPLE  PROCESSORS  59 

SDPPCPT  OP  INPUT/OUTPUT  60 

SUPPORT  FOR  THE  INITIAL  SECURE  STATE  63 


SECTION  V 


REQUIREMENTS  FOR  THE  PROTECTED  SYSTEM 


64 


INTRODUCTION 
SPM  INSTRUCTIONS 

PROCESSOR  REALIZATION  REQUIREMENTS 
Required  Certified  Oo^arations 
REALIZATION  REQUIREMENTS  - MAIN  MEMORY 
Error  Detection  and  Correction 
Read- Alter-Write  Memory  Operation 
Module  Identification 

Rapid  Standardization  of  Memory  Contents 
REALIZATION  REQUIREMENTS;  PERIPHERAL  CONTROLLERS 
AND  DEVICES 

CACHE  MEMORY  REALIZATION  CONSIDERATIONS 
Cache  Size 

Descriptor  Bits  Not  Mandatorily  Represented  in  the 
Cache 


64 

64 

72 

73 

74 
74 

74 

75 
75 

75 

76 

77 

77 


SECTION  VI  IMPLEMENTATION  CONSIDERATIONS 


79 


INTRODUCTION  79 
ESTIMATES  OF  SIZE  AND  COST  80 
TRADE-OFF  NOTES  84 
RELIABILITY  CCNSIDEPATIONS  85 
DESIGN  RECOMMENDATIONS  - RELIABILITY  93 
PEPFCRMANCE  ESTIMATES  94 
SUMMARY  - EXPECTED  PERFOPMANCE  101 
PERFCRMANCE  ANALYSIS  - SUMMARY  101 

APPENDIX  A AF  COMMENTS  102 


REFERENCES  104 


LIST  OF  ILLDSTFATIONS 


Fiqur?  Nutrbgr 

1 The  Beference  Monitor 

2 The  SPF 

3 Mapping 

li  Descriptor  Structure 

5 Rings 

6 Ring  Brackets 

7 Levels  of  Abstraction 

8 Levels  of  Abstraction  and  Rings 

9 Outward  Ring  Crossing 

10  Kernel  Structure 

11  Process  Execution  States 

12  Host  Coirputer 

13  Host  Computer  Kith  SPM 

1i*  SPK  Block  Eiaqram 

15  Virtual  Address 

16  BDR  Layout 

17  Descriptor  Formats 

18  Multicrocessor  System 

19  Size  vs.  Address  Width 

20  Estimated  Performance  Reduction 

21  Effective  Memory  Speed  Loss  Due  To  Descriptor  Cache 

Delay 


Paae 

r 

11 
. 16 
24 
27 

30 

31 

33 

34 

35 
37 
43_ 

47 

48 
51 

55 

56 
' 57 

61 

82 

98 

100 


Table  Nymbey  Page 


Table  Nymbey  Page 

I Fault  and  Trap  Information  i 69. 

II  Security  Related  Fault  Information  71 

III  Reliability  Function  for  Independent  Fault  Detection  S8 

IV  Analyses  Required  91 


r 


” 1 


Acknowledgement 


This  document  is  an  edited  version  of  a technical  report  produced  by 
Honeywell  Information  Systems,  Inc.  under  contract  P19628-74-C-0205. 
While  the  iraiority  of  the  work  was  done  by  Honeywell,  extensive 
revisions  have  teen  made  to  reflect  the  editorial  comments  of  project 
members  at  ESD/MCIT  and  MITRE.  The  editorial  comments  were  provided 
by  Haj.  R.  H.  Schell,  Capt.  K.  R.  Price,  Capt.  P.  A.  Karqer  of 
ESD/MCIT,  and  K.  J.  Biba  of  MITRE.  The  authors  for  Honeywell  were 
John  Mekota,  John  Gilson,  and  many  others. 


r 


SECTION  I 
INTRCDUCTICN 

PUPPCSE  OF  STUD! 


"he  obiectivfes  of  this  study  were  to  detail  the  architectural 
requiretrents  for  a Secure  Conirunications  Processor  and  to  study  the 
dosiqn  trade-offs  in  the  specification  of  such  a processor.  The 
Secure  Communications  Processor  is  needed  for  many  diverse  military 
applications,  ircludinq; 


* As  a front  end  processor  for  a secure  general  purpose  computer 
system 

* As  an  interface  message  processor  connecting  a general  purpose 
computer  to  a computer  network  environment 

* As  a node  in  a store-and-forward  or  packet-switched 
communicatiors  network 

* As' a stand  alone  communications  terminal  processor 

* As  a switching  processor  for  dynamic  reconfiguration  of  large 
secure  general  purpose  computer  systems. 

The  architecture  was  derived  from  fundamental  security  design  concepts 
that  are  demonstrably  sufficient  to  ensure  the  effectiveness  of 
internal  hardware/software  security  controls. 


PACKGRCUND 


The  military  and  civilian  branches  of  the  0.  S.  Government  are  heavy 
users  of  data  processing  equipment.  This  generates  much  economic 
pressure  to  obtain  the  most  efficient  computing  arrangements  possible, 
including  sharing  of  facilities  by  the  maximum  number  of  government 
users  and  applications.  Moreover,  many  of  the  uses  of  computers,  such 
as  in  data  networks,  command  and  control,  and  message  switching 
systems,  inherently  require  the  facilities  to  be  shared  by  many  users 
in  order  to  perform  their  functions,  ether  applications  such  as  data 
concentrators,  satellite  computers,  etc.  have  communications 
characteristics  in  the  routing  and  transmittal  of  information. 

Ker.ee  the  Government  has  m.uch  interest  in  multi-user  computer  systems 
for  maximizing  the  value  obtained  from  each  tax  dollar.  (Reference  1) 
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However,  inuch  of  this  work  involves  processing  information  of  widely 
different  security  levels  (1)  and  categories,  (2)  generated  by  many 
different  proiects  in  many  different  agencies  and  branches,  which  must 
be  kept  carefully  segregated  lest  highly  sensitive  information  reach 
the  wrong  hards.  Current  multiuser  computer  systems  are  fundamentally 
inadequate  tc  provide  protection  for  the  information  they  process. 
While  current  systems  do  an  excellent  iob  of  keeping  non-malicious 
users  from  interfering  with  one  another,  the  consistent  success  of 
"Tiger  Teams"  in  penetrating  these  systems  shows  that  they  are 
vulnerable  to  the  attack  of  knowledgeable,  patient,  and  malicious 
users.  See  References  3U  and  35.  The  least  vulnerable  is  the  Multics 
system,  currently  under  security  development  sponsored  by  the  USAF. 
However,  Kultics  is  several  orders  of  magnitude  too  large  and 
expensive  for  the  applications  contemplated  here. 

There  is  little  risk  (except  for  denial  of  service)  in  a processor 
that  manipulates  only  the  address  or  header  information  of  m;assages  in 
the  clear,  with  th®  body  of  the  message  encrypted.  The  security  of 
the  information  is  safeguarded  by  the  encryption.  A hostile  agent  who 
accessed  the  encrypted  data  would  have  to  break  the  code  to  get  access 
to  the  information.  The  encrypted  version  would  be  more  easily 
accessible  to  a hostile  party  through  tapped  communication  circuits 
rather  than  computer  penetration.  The  risk  becomes  large  when  the 
entire  massage  is  handled  in  the  clear  and  prohibitively  large  when 
the  processor  is  used  for  other  tasks,  concurrent  with  message 
handliiiq.  The  increasing  capabilities  of  small  processors,  brought 
about  by  technological  advances  in  speed  and  memory  capability, 
produce  pressures  to  use  a single  processor  both  as  a message  handler 
and  also  as  a processor  of  the  massages  handled.  Such  usage  increases 
the  risk  of  compromise  and  leads  to  a need  for  a small  Secure 
Communications  Processor. 


(1)  security  levels  in  this  context  refers  to  the  DoD  classif ication 
system  for  ccntrollinq  access  to  defense  sensitive  information. 
Security  levels  are  applied'  to  instances  of  information  and  DoD 
clearances  are  granted  by  level  for  basic  permission  tc  access  such 
information.  These  levels  include  Confidential,  Secret  and  Top 
Secret . 


(2)  security  categories  in  this  context  refers  to  further  refinement 
of  control  of  access  to  defense  sensitive  information.  Access 
requires  the  person  to  hold  a sufficient  security  clearance  level  and 
also  have  the  proper  formal  category  designation  to  use  the 
information-  Categories  are  normally  associated  with  the  type  of 
information.  Examples  of  categories  include  NATO,  Nuclear,  and 
Crypto. 
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Use  of  a communications  processor  as  a front-end  for  a large  nultiuser 
system  processing  data  at  several  security  levels  presents  an  inviting 
target  for  a malicious  user.  The  risks  include  both  the  risk  of  a 
user  acquiring  or  modifying  information  to  which  he  has  no  access 
authorization  and  the  possibility  of  a malicious  user  denying  use  of 
the  system  at  a critical  time.  In  military  use,  the  value  of  the 
information  acquired  or  modified  or  the  gain  achieved  by  denying  use 
of  the  system  may  warrant  virtually  any  level  of  effort  to  achieve  the 
cenetr ation. 

The  Air  Force  has  shown  in  earlier  work  (Reference  7)  that  a secure 
comouter  system  can  be  constructed  using  the  concept  of  the  reference 
monitor.  A reference  monitor  is  a hardware-software  mechanism  that 
controls  the  access  of  subiects  (active  system  elements)  to  objects 
(urlts  of  information)  within  the  computer  system.  Figure  1 presents 
a schematic  diagram  of  the  relation  among  subjects,  objects,  reference 
monitor,  and  reference  monitor  authorization  data  base  (access 
matrix) . See  Reference  22.  The  figure  gives  examples  of  typical 
subjects,  objects,  and  data  base  items. 

In  operation,  the  reference  monitor  allows  or  forbids  access  by 
subiects  to  objects,  making  its  decisions  on  the  basis  of  subject 
identity,  object  identity,  and  security  parameters  of  the  subject  and 
object.  The  reference  monitor  both  mechanizes  the  access  rules  of  the 
military  security  system  and  assures  that  they  are  enforced  within  the 
computer.  An  implementation  of  the  reference  monitor  is  termed  a 
"security  kernel". 

A reference  monitor  must  meet  the  following  three  requirements  in 
order  to  provide  the  basis  for  a multilevel  secure  computer  system: 

a-  Completeness  - the  reference  monitor  must  be  invoked  on  every 
access  by  a subject  to  an  object; 

b.  Isolation  - the  reference  monitor  and  its  data  base  must  be 
protected  from  unauthorized  alterations; 

c.  Certif iability  - the  reference  monitor  must  be  small,  simple 
and  understandable  so  that  it  can  be  tested  and  verified  to 
perform  its  functions  properly. 

Both  the  requirement  for  completeness  and  that  for  certif iability 
demand  that  the  reference  monitor  include  hardware  as  well  as  software 
- the  form.er  because  software  validation  of  every  access  by  a subject 
to  an  object  would  add  intolerable  complexity  and  overhead  to  the 
reference  monitor,  the  latter  because  certain  hardware  architectures 
preclude  the  construction  of  a simple  understandable  operating  system. 
The  purpose  of  this  study  is  to  define  a suitable  architecture. 
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?®coqnizinq  the  importance  of  the  ideal  model  of  a reference  monitor 
to  achieving  computer  security,  the  Air  Force  has  also  developed 
several  mathematical  models.  After  preliminary  efforts  (Reference  2) 
both  a finite  state  model  (Reference  3)  and  a level  of  abstraction 
model  (Reference  U)  have  been  prepared  and  are  currently  being 
refined.  The  models  have  provided  structure  and  guidance  for  this 
study  effort.  They  contain,  or  directly  imply,  security  guidelines, 
principles,  and  theorems  used  extensively  in  this  study. 

It  is  necessary  to  verify  that  any  implementation  of  a security  kertiel 
is  done  correctly,  in  order  to  meet  the  certif iability  requirement  for 
a reference  monitor.  The  Air  Force  has  also  developed  several 
approaches  to  this  verification  requirement.  Both  the  levels  of 
abstraction  model  (Reference  4)  and  the  work  of  Price  (Reference  5) 
aive  guidance  for  meeting  the  certif iability  requirement. 

The  Air  Force  has  developed  a prototype  security  kernel  using  a 
PEP-11/45  (References  fi,  7).  The  results  of  this  effort,  particularly 
the  performance  impacts,  were  considered  carefully  in  this  study.  The 
Air  Force  is  also  sponsoring  development  efforts  leading  to  the  design 
of  a security  kernel  for  the  Multics  system.  The  lessons  learned  in 
that  effort  were  also  applied  to  this  study. 


TECHNICAL  APFRCACH 

The  security  models  specify  the  secure  operation  of  a system  composed 
of  sublects  and  oblects-  A security  kernel  must  implement  both  the 
rules  of  the  model  and  the  subjects  and  objects  that  these  rules 
control.  The  usual  interpretation  of  subject  is  that  of  a process 
(exfecutior  of  a program.,  see  Reference  23.)  The  usual  interpretation 
of  an  object  is  a virtual  memory  segment.  Provision  of  a true 
reference  monitor  requires  complete  implementation  of  all  aspects  of 
the  model. 

The  implementation  of  subjects  and  objects  is  constrained  by  the 
hardware  on  which  the  security  kernel  operates.  If  the  hardware  does 
not  facilitate  the  simple  implementation  of  subjects  and  objects,  the 
certifiability  requirement  for  a reference  monitor  will  not  be  met.  A 
minimum  requirement  for  secure  computer  systems  is  the  use  of 
descriptor  driven  processor  architectures  that  implement  segmented 
memories.  (3)  With  such  processors,  the  objects  of  the  model  can 


(3)  A descriptor  driven  processor  is  one  whose  hardware  interprets 
each  "virtual"  address  issued  by  a program  in  terms  of  a set  of 
descrictors  that  specify  the  real  physical  address  and  permitted 
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correspond  to  seqments  supported  by  the  hardware.  R properly 
organized  seqtrented  memory  merges  primary  storage  (core  or 
sem.iconductor  RAM)  and  secondary  storage  (bulk  store,  disk,  peripheral 
devices)  maragemert  functions,  eliminating  from  security  consideration 
any  separate,  complex,  and  security  related  "file  system".  Further, 
the  sublects  of  the  model  correspond  to  processes  (address 
space-processor  state  pairs)  supported  directly  by  descriptor  driven 
processor  hardware. 

The  security  kernel  software  defined  by  the  model  and  implemented  on 
descriptor  driven  hardware  is  a simple  mechanism  that  implements  only 
the  security  rules,  the  sub-jects,  and  the  objects.  It  does  not 
provide  the  full  facilities  of  an  operating  system;  it  could  not  do 
so  without  developing  so  much  complexity  that  it  would  no  longer  be  a 
prototype.  Instead,  the  complex  functions  required  of  an  operating 
system  are  provided  by  programs  external  to  and  controlled  by  the 
kernel.  These  functions  can  be  arbitrarily  complex,  but  are  not 
security  related.  However,  some  may  be  sensitive  in  terms  of  assuring 
the  smooth  operation  of  the  computer  system.  For  exam.ple,  a typical 
operating  system  (not  kernel)  function  such  as  a scheduler  which  has 
no  access  *-o  user  information  cannot  compromise  security,  but  it  can 
slow  service  to  users.  (U)  Denial  of  service  is  not  a security  issue. 


To  assure  that  user  programs  can  be  separated  from  (and  kept  from 
interfering  with)  such  sensitive  programs,  the  previous  development 
efforts  in  multilevel  security  have  identified  the  need  for  hardware 
with  at  least  three  separate  domains  of  execution  (states  of  program 
privilege).  Cf  these,  one  can  be  allocated  to  the  security  kernel, 
the  second  to  the  operating  system,  and  the  third  to  user  programs. 

The  security  kernel  can  then  protect  the  operating  system  ftom  user 
programs  and,  with  suitable  architectural  organization  for  the 
hardware,  the  transitions  from  one  domain  to  another  can  be  rapid  and 
efficient.  Thus  the  technical  approach  for  this  study  is  based  on  the 
concept  of  a security  kerrel,  using  a model  of  the  kernel  that 
represents  the  secure  operation  of  an  ideal  reference  monitor.  The 
ideal  reference  monitor  requires  that  ♦■he  kernel  implement  subjects. 


access  modes  (e.g.,  read,  write,  execute)  to  be  associated  with  every 
possible  "virtual"  address. 

(U)  Malicious  tampering  with  the  scheduling  of  a process  or  operation 
cannot  give  a process  access  to  forbidden  data,  only  the  security 
kernel  can  change  data  access  rights.  The  scheduling  algorithm  can 
prevent  an  authorized  process  from,  access  to  a processor  and  hence 
prevent  that  process  from  operating  at  all. 
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obiects,  and  the  security  rules  qoverninq  their  interactions.  Siirple 
(certifiable)  irrplementation  hinqes  on  development  of  an  architectural 
desiqn  tnat  supports  these  functions.  Prior  work  (Reference  24)  has 
shown  that  a secure  computer  system  must  supply  a seqmented  virtual 
memory,  controlled  I/C  facilities,  at  least  two  processor  domains,  and 
requires  use  of  descriptor  driven  hardware.  The  above  considerations 
and  extensive  experience  and  study  of  the  prior  work  were  combined 
with  qood  enqineerinq  judqement  to  make  the  numerous  desiqn  trade-offs 
required  here. 


DESIGN  ESQ'JIPEMENTS 

The  work  reported  here  attempted  to  cover  all  the  necessary  studies 
and  analysis  necessary  to  define  an  architecture  for  a secure  and 
effective  communications  processor.  Specific  considerations  included; 

* The  iirplementaion  of  the  reference  monitor  concept  or  a 
functional  equivalent  is  required.  The  development  of  a security 
kernel  to  implement  the  reference  monitor  is  one  alternative. 

* Isolation  of  the  security  kernel  from  the  remainder  of  the  system 
must  be  provided  by  use  of  at  least  three  hierarchically  ordered 
protection  domains.  Both  rinq  mechanisms,  as  proposed  by 
Schroeder  (reference  8)  and  used  in  Hultics,  and  capability  based 
domain  mechanisms  as  suqqested  by  Price  and  others  (Reference  5) 
were  considered. 

* A virtual  addressinq  mechanism  usinq  descriptor  based  addressing 
must  nrovide  the  potential  for  relatively  large  number  of 
independent  segments.  A true  two  dimensional  addressinq  facility 
with  independent  access  controls  (read,  write,  execute,  etc)  for 
each  seqraent  is  recommended. 

* A mechanism  must  be  provided  for  rapidly  switching  between  the 
descriptor  sets  for  various  processes,  in  order  to  allow 
efficient  context  switching. 

* The  design  must  ensure  that  those  mechanisms  actually 
iiTpleTentinq  security  controls  shall  be  well  defined  and  well 
isolated  from  other  elements  of  the  processor. 

* The  isolation  of  the  implementing  mechanisms  must  include  any 
component  which,  if  subject  to  malicious  tampering  and/or  random 
hardware  failure,  could  invalidate  the  security  controls. 

* Input  and  output  (I/C)  operations  must  be  subject  to  direct 
security  controls  since  I/C  constitutes  exchange  of  data  with 


. oblects.  Both  the  techniques  of  virtualized  I/O,,  proposed  by 
Clark  (Beference  9)  and  iirplementation  of  individual  I/O  devices 
as  parts  of  the  virtual  ireirory  of  processes  operatinq  outside  of 
the  security  kernel  were  considered. 

* The  resulting  Secure  Coirirunications  Processor  must  be  capable  of 
being  iirplemented  to  meet  the  demands  of  various  military 
specifications  including  adverse  environmental  conditions  and 
airborne  applications. 


HECCMKZNCEJ  DESIGN 

Various  approaches  to  crovidinq  a hardware  base  for  the  reference 
monitor  were  considered.  The  most  attractive  one,  from  the  viewpoint 
of  the  systems  architect,  is  a completely  new  design.  This  allows 
iraximurr  freedom  to  provide  an  optimum  solution  of  the  requirements, 
and  gives  miirimum  constraints.  Unfortunately,  it  also  tends  to 
maximize  development  costs,  lead  time,  and  support  costs.  The 
alternative  of  modifying  an  existing  minicomputer  (or  family  of 
minicomputers)  was  also  considered. 

Consideration  of  the  total  cost  and  effort  needed  to  provide  a working 
communications  system  lent  great  weight  to  the  modification  approach. 
The  effort  to  produce  system  and  user  software  and  engineering 
documentation  for  testing,  training,  and  miaintenance,  so  far  outweighs 
the  effort  to  produce  hardware  that  the  trade-offs  were  heavily 
weighted  towards  an  approach  that  would  allow  use  of  most  existing 
software  and  engineering  documentation.  In  an  effort  to  provide  a 
minimum  cost  solution,  several  different  base  systems  including  those 
of  other  manufacturers  were  studied. 

The  result  of  the  study  was  selection  of  a unique  architectural  design 
for  the  Secure  Communications  Processor.  Consideration  of  the  design 
requirements  listed  above  and  examination  of  both  current  and 
anticipated  computer  system  designs  lead  to  the  use  of  a Security 
Protection  lodule  (SPM) . The  Security  Protection  Module  is  a new 
Piece  of  hardware  that  will  be  logically  and  physically  interposed 
between  the  components  of  a standard,  commercially  available  computer 
system.  Figure  2 shows  the  use  of  the  Security  Protection  Module. 

Note  the  similarity  to  the  ideal  reference  monitor  shown  in  Figure  1. 

The  SPK  will  provide  a direct  realization  of  the  security  kernel  inodel 
since  its  location  between  the  active  units  (central  processor, 
memory,  I/n  controller,  etc)  of  the  protected  computer  allows  meeting 
the  first  ♦wo  requirements  for  a reference  monitor.  Complete 
mediation  is  ensured  since  all  references  must  pass  through  the  SFM. 
Isolation  is  ensured  by  use  of  a separate  item  of  hardware.  The  final 
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requirement,  siirplicitv,  may  be  attained  since  the  SPM  has  only  to 
consider  the  security  rules  of  the  model.  Only  aspects  requiring 
Dhysical  isolation  or  performance  assistance  are  supported  in  the 
hardware.  The  protected  computer  provides  the  operating  system  and 
application  programs. 

An  SP!i  connected  in  this  way  could  potentially  be  used  with  many 
different  computers  at  different  levels  of  complexity,  cost,  and 
performance.  The  requirements  placed  on  the  computer  are  detailed 
later.  Use  of  an  SPM  could  allow  an  otherwise  standard  computer  to 
serve  as  a Secure  Communications  Processor.  There  would  he  need  for 
special  documentation,  training,  and  support  only  for  the  SPM  and  its 
related  software,  with  the  remainder  of  the  computer  receiving 
standard  support.  This  is  particularly  attractive  for  the  development 
efforts  as  only  the  SPM,  the  security  kernel  software,  and  the 
interfaces  need  be  proved  correct  in  accordance  with  the  security 
model. 

Use  of  an  SPM  could  severely  degrade  the  performance  of  the  system  by 
adding  time  to  each  action  of  the  protected  central  processor.  A 
considerable  amount  of  the  design  effort  was  devoted  to  avoiding  or 
mirimizinq  such  degradation.  Estimates  indicate  that  performance 
degradation  of  less  than  twenty  five  percent  should  oe  realizable  even 
if  the  protected  processor  already  takes  maximum  auvantage  of 
available  technology.  Some  combinations  of  protected  processor  and 
SPM  design  could  have  performiance  degradations  of  under  half  this 
amount,  and  it  is  possible  to  design  an  SPM  which  would  actually 
improve  performance  of  many  unprotected  minicomputers. 

The  isolation  of  the  Security  Protection  Module  and  the  close  relation 
between  the  ideal  reference  monitor  model  and  the  Security  Protection 
Module  should  make  the  task  of  verification  or  certification  easier. 
The  design  of  the  SPM  has  been  formulated  with  the  guidance  of  the 
models.  Continuing  this  pattern  should  result  in  an  SPM  that  can  be 
certified  secure. 


SUMHA3Y  AND  RF.COMMENDATICNS 

The  problem  of  specifying  an  architecture  for  a Secure  Communications 
Processor  was  studied  in  light  of  the  previous  work  cn  computer 
security.  A particular  architecture  was  selectee  and  the  requirements 
applied  against  it.  The  result  is  the  SPM  centered  architecture 
introduced  above  and  described  in  greater  detail  later.  A functional 
specification  fer  the  SPM  was  also  prepared. 

Much  remains  to  be  done.  The  general  concept  of  the  SPM  should  be 
refined  and  expanded  to  a detailed  specification.  A prototype  SPM 
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should  be  fabricated  for  use  with  a specific  computer  system.  This 
prototype  should  be  tested  and  evaluated  to  determine  the  utility  of 
the  SPM  approach.  The  verification  and  certification  efforts  should 
also  continue  as  the  project  moves  towards  production  of  a certifiable 
Secure  Communications  Processor. 


i 
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The  concept  of  a reference  monitor  is  well  established  in  the  computer 
security  field  .for  both  theoretical  and  practical  applications.  k 
realization  of  a reference  monitor  is  called  a security  kernel  and  is 
expected  to  be  specific  to  a certain  computer.  The  Security 
Protection  Module  proposed  here  could  provide  hardware  support  for  a 
security  kernel  for  a qeneral  class  of  computer  systems. 

An  SPd,  combined  with  the  software  required  to  drive  it,  and  the 
connections  to  the  protected  computet  system,  can  function  as  a 
security  kernel  and  thus  as  a reference  monitor.  It  is  currently 
estimated  that  20-30%  of  the  capability  will  be  in  hardware  with  the 
rest  in  software.  Functions  have  been  placed  in  the  SPrt  hardware  when 
y a practical  system  requires  hardware  support  for  adequate  performance 

and  efficient  operation.  The  remaining  functionality,  orimarily  that 
of  control  of  the  hardware  has  been  assiqned  to  the  software. 


MODELS 


"he  qoverninent  has  devoted  much  attention  to  the  development  of 
xathematicai  models  of  computer  security.  This  effort  is  essential  to 
the  development  of  a certifiable  reference  monitor.  To  be  certified 
secure,  the  implemented  computer  and  supportinq  software  must  be 
proved  to  conform  to  the  model  of  security  certified  correct  by  DoD. 
The  mathematical  models  produced  (References  1,  2,  3,  4,  etc)  were 
considered  in  this  study. 

Modelinq  efforts  have  identified  numerous  requirements  for  building 
secure  systems  and  have  codified  the  security  preservation  rules.  The 
models  define  the  fundamental  notions  with  precision  and  give  rules  of 
operation  for  keeping  a system  secure  once  it  has  been  initialized  as 
secure. 


The  models  deal  with  sublects  and  ob-jects.  Figure  1 shows  that 
subdects  are  the  active  entities  in  the  system,  such  as  processes 
acting  for  users  and  processes  executing  for  system  purposes.  Objects 
are  the  passive  entities  in  the  system,  such  as  data  and  program 
segments  in  memory,  machine  registers,  and  peripheral  devices.  The 


models  define  the  types  of  access  that  a subject  may  have  to  an  object 
under  all  conditions.  The  access  allowed  depends  on  the  contents  of 
the  reference  monitor  authorization  data  base  which  identifies  the 
clearances,  classifications,  and  need-to-know  attributes  of  both 
subjects  and  objects. 

The  model  requires  that  the  reference  monitor  authorization  data  base 
contain  the  current  access  relationships  between  all  subjects  and 
objects.  The  data  base  must  identify  three  elements:  the  identifier 
of  the  subject,  the  identifier  of  the  object,  and  the  allowed  access 
mode  (read,  write,  execute)  for  the  subject  to  access  the  object. 

The  reference  monitor  data  base  must  contain  the  security  level  of 
each  subject  and  object.  This  is  used  alonq  with  the  mode  to  grant 
access  of  subjects  to  objects.  A security  level  consists  of  two 
components,  a classification  and  a set  of  access  categories. 
Classifications  or  clearances  form  a hierarchy  with  higher 
classifications  having  access  to  all  lower  classifications.  The 
simple  security  condition  reguires  that  a subject  have  a clearance 
level  egual  to  or  greater  than  that  of  an  object  before  read  access  to 
that  object  can  be  granted.  Categories  are  not  hierarchically  ordered 
but  are  ordered  by  set  inclusion.  The  simple  security  condition 
allows  subject  to  have  read  access  objects  only  If  the  subject  has  all 
categories  of  the  object  (it  may  have  more).  A subject  may  not  access 
an  object  if  the  object  has  any  category  that  the  subject  lacks. 

In  addition  to  the  simple  security  condition  it  is  necessary  to 
enforce  the  •’♦-property*'  (pronounced  "star  property")  (Reference  3), 
The  simple  security  condition  allows  a subject  read  access  to  objects 
only  with  a security  level  less  than  or  egual  to  the  security  level  of 
the  subject.  The  ♦-property  allows  a subject  to  have  write  access 
only  to  objects  with  a security  level  egual  to  or  greater  than  that  of 
the  subject  (and  with  the  proper  categories).  These  properties  are 
sometimes  stated  as  preventing  a subject  from  "reading  up"  (reading 
material  with  a classification  greater  than  that  of  the  subject)  and 
from  "writing  down"  (writing  material  at  a classification  level  less 
than  that  originally  assigned  to  the  material). 


The  combination  of  classification  and  category  set  gives  the  security 
level  of  the  subject  or  the  object.  Possible  relations  between  two 
security  levels  include  greater  than,  less  than,  equal  to  and  isolated 
from. 

Mniform  application  of  the  simple  security  condition  and  the 
♦-property  rules  results  in  a system  that  is  secure  but  not  useful. 
Some  subjects  must  be  able  to  violate  the  property.  These  are  called 
"trusted  subjects".  In  particular,  the  "System  Security  Officer"  is  a 
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The  model  of  a secure  system  shows  the  reference  monitor  must  possess 
three  characteristics; 

* Complete  irediatior 

* Isolation 

* Provable  correctness 

The  architecture  must  fully  support  these  three  characteristics  as 
well  as  orovidinq  the  necessary  facilities  to  perform  the  required 
functions  of  the  applications  the  secure  architecture  must  support. 
The  two  most  important  of  these  secondary  characteristics  are 
efficiency  and  facility  for  information  sharing.  If  a system  is  not 
“fficient  no  real  work  can  be  supported.  If  information  cannot  be 
easily  shared,  the  system  might  as  well  be  a single  level  of  security 
system. 

Complato  Mediation 


The  requirement  for  complete  mediation  of  the  reference  monitor  can  be 
achieved  in  several  ways.  The  reference  monitor  could  use  software  to 
achieve  all  mediation.  This  would  require  up  to  an  order  of  magnitude 
increase  in  the  instruction  execution  rate  since  each  address 
reference  would  have  to  be  checked  for  validity  by  a software  routine. 
This  is  clearly  too  inefficient  to  be  practical.  There  are  two  ways 
that  hardware  could  mediate  the  access  for  the  reference  monitor.  The 
first  might  be  called  the  static  case.  In  the  static  case  the  memory 
space  would  be  allocated  statically.  Each  allocation  would  be  checxed 
for  access  privileges  at  allocation  time.  At  the  time  of  actual 
access  the  reference  would  only  have  to  insure  that  the  hardware 
I performed  correct  limit  checking  to  see  that  the  access  did  not  exceed 

■ the  boundaries  of  the  allocation  (l^eference  25).  The  limitations  of 

this  system  require  that  all  information  sharing  be  performed  through 
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the  interprocess  coirmunicat ion  channel.  This  requires  that  the 
inforiration  by  physically  copied  twice  for  each  act  of  coinirunication. 
This  imposes  an  intolerable  burden  on  the  efficiency  of  the  operation. 

The  other  malor  hardware  technique  that  can  be  employed  is  the 
iirple mentation  of  a descriptor  based  virtual  memory  system.  Each 
loqical  obiect  is  defined  as  a separate  virtual  memory  segment  with  a 
descriptor  giving,  the  access  controls.  The  reference  monitor  set  the 
descriptors  and  gives  access  to  subjects  with  the  descriptors.  The 
hardware  mast  check  via  the  descriptors  for  each  access  to  real 
memory.  If  this  application  of  the  descriptors  is  implemented  through 
hardware  properly,  the  efficiency  of  the  wnole  process  can  be  made 
sufficiently  high  to  support  the  needed  systems  capaailities. 

.'lediatior.  must  be  performed  both  for  cpu  access  and  for  I/O  as  well. 

In  the  virtual  memory  system,  the  1/0  devices  which  respond  directly 
to  commands  tv  transferring  a unit  of  data  at  a time,  can  be  treated 
as  objects.  Those  which  transfer  blocks  of  data  by  .direct  memory 
access  must  be  treated  as  subjects  with  given  security  capabilities. 


Isolation 


To  enable  the  reference  monitor  to  be  securely  isolated  from  all  of 
the  other  programs  in  the  system,  the  system  must  provide  memory 
protection  to  the  reference  monitor  and  provide  it  with  a set  of 
privileged  instructions  which  allow  the  reference  monitor  alone  to 
control  the  creation  and  assignment  of  descriptors.  The  necessity  for 
memory  protection  and  special  privileged  instructions  is  common  to  all 
systems  of  implementation  tor  the  reference  monitor. 


Provable  Correctness 


7or  the  system  to  be  certifiably  secure  it  is  necessary  to  prove 
mathematically  that  the  system  observes  the  rules  of  the  security 
model  under  all  conditions.  This  means  that  each  step  from  the  model 
through  formal  specifications  to  algorithms  and  finally  to  computer 
hardware  and  software  must  be  provable.  The  key  to  proving  the 
correctness  of  the  specifications,  algorithms  and  programs  is  rigor 
and  simplicity.  The  proofs  must  obey  a riqoLOus  formalism  to  be 
proved  mathematically.  The  computer  hardware  and  the  requisite 
software  to'  implement  the  reference  monitor  must  be  simple  and 
straightforward. 
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There  are  several  efficiency  considerations  that  must  be  addressed. 

The  sharinq  of  information  between  subject  has  beer  mentioned.  This 
is  a primary  consideration.  If  sharing  is  not  an  objective,  the 
secure  computing  can  be  provided  by  several  single  level  machines, 
uri-proq rammed  with  complete  cleansing  between  users  if  necessary. 

Another  efficiency  consideration  is  the  overhead  that  the  reference 
monitor  imposes  on  the  exchange  of  processor  between  multiple  active 
processes  (subjects) . If  there  is  too  much  overhead,  then  the  sharinq 
of  the  system  by  several  subjects  is  severely  impaired.  This  overhead 
is  less  visible  than  the  more  common  concern  of  the  overhead  involved 
with  the  access  by  a single  process  to  several  descriptor  based 
virtual  memory  segments  as  mentioned  above. 


.'1AFPIN3  ADDRESSES 

One  wav  for  a security  kernel  to  perform  its  function  cf  complete 
mediation,  that  is,  of  checking  each  reference  of  a subject  to  an 
object,  is  by  providing  a mapping  function.  In  this  view,  the  kernel 
maps  (or  translates)  requests  made  by  a subject  into  approved  requests 
transmitted  to  an  object.  The  central  position  of  the  SPM  makes  it 
possible  to  enforce  such  mapping. 

The  subject  (process)  addresses  its  resources  (objects  such  as  memory 
ard  peripheral  devices)  by  names  known  to  the  subject  and  the  kernel, 
commonly  called  "virtual"  addresses.  The  kernel  approves  each  request 
and  maps  (or  translates)  the  virtual  address  tc  a physical  address. 
Physical  addresses  are  known  to  the  kernel  and  the  objects. 

Figure  3 shows  this  mapping  function  of  the  SPM.  The  subjects,  which 
are  processes  in  operation,  deal  with  virtual  addresses  for  both 
memory  and  peripheral  devices.  The  SPM  checks  these  references  by  the 
security  preservation  rules  and  translates  or  maps  the  virtual  address 
into  a physical  address.  This  mapping  function  is  similar  to  that 
performed  in  the  Multics  system.  The  map  can  carry  both  translation 
information  and  much  of  the  information  from  the  security  kernel  data 
base. 

'•apping  memory  addresses  from  virtual  to  absolute  form  is  well 
understood  and  involves  few  complications.  More  elaborate  memory 
management  assistance,  such  as  paging,  may  be  incorporated  into  the 
hardware  without  introducing  new  kinds  of  architectural  problems. 
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Figure  3 
Mapping 


flapping  peripheral  addresses  from  virtual  to  alsolute  form  is  an 
extension  of  the  maopinq  function  to  enforce  security  on  peripheral 
operations.  There  are  several  ways  to  handle  the  representation,  the 
peripheral  devices  exhibit  wide  variation  in  characteristics,  and  the 
resultinq  data  exchanqa  may  extend  over  lonq  periods  of  time.  Two 
ways  to  handle  the  mapping  ate  of  particular  interest  here,  the  use  of 
explicit  names  for  devices  and  the  consideration  of  devices  as  part  of 
memory. 

In  the  consideration  of  devices  as  memory,  the  subjects  access  the 
peripheral  devices  in  exactly  the  same  way  as  other  objects.  Each 
object  has  a virtual  address  and  access  capabilities.  The  SPM  , 
enforces  the  access  limitations  during  the  virtual  address  to  absolute 
address  translation  process.  The  operation  of  1/C  in  this  manner  is 
the  technique  used  in  the  PDPrII  Unibus  architecture. 

The  problem  with  this  approach  arises  with  Direct  Kemory  Access  (B.'IA) 
type  devices.  These  devices  act  like  subjects  as  they  access  data 
segments  to  transfer  their  data.  There  are  two  choices  for  these 
devices,  they  can  be  given  identifications  as  resources,  i.e,  set  up 
as  recognized  processes,  and  provided  with  hardware  connection  to  the 
SEK  to  allow  their  addresses  to  be  translated  by  the  SF1  on  the  same 
basis  as  those  of  a CPU  or  they  may  be  given  absolute  (pre-mapped) 
addresses  and  then  must  be  certified  to  operate  correctly,  i.e. 
certified  secure  and  treated  as  trusted  processes,  logically  in  the 
kernel  domain. 


hap  ?*ar! ipu latiors 

The  mapping  functions  must  be  maintained  in  a form  accessible  only  to 
the  security  kernel  to  meet  the  requirement  of  isolation.  The 
security  kernel  must  be  able  to  manipulate  the  maps  in  order  to 
reflect  changes  in  the  operating  state  of  the  system.  Manipulations 
required  include: 

♦ generating  a resource  description  and  placing  it  in  a map 

♦ altering  a resource  description  in  a map 

♦ removing  a resource  description  from  a map 

Examples  are  assigning  a block  of  memory  (core  or  secondary) , 
assioning  a peripheral  device,  extending  the  size  of  a memory  block, 
or  releasing  a memory  block  or  peripheral  device. 
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ffap  Tl^aalization  - Descriptors 


Various  schemes  that  provide  for  memory  segmentation  and  paging  have 
been  known  for  many  years  (References  26,  27,  28,  and  29),  Based  on 
oast  experience  and  current  thinking,  a a method  has  been  developed 
which  directly  implements  the  abstract  model  of  mapping.  This  is  the 
hardware  descriptor,  as  exemplified  by  the  Honeywell  level  68 
processor  used  by  the  Multics  system.  The  descriptor  mechanism  is 
essentially  a formalized  hierarchy  of  pointer  tables.  References  by 
subiects  (virtual  addresses)  are  transformed  to  object  references 
(absolute  addresses)  through  one  or  more  levels  of  descriptor.  Figure 
h shows  tnis  in  skeleton  form. 

Descriptor  references  perform  the  mapping  of  virtual  to  absolute 
addresses  required  by  the  model.  Descriptors  may  be  cascaded,  with 
one  pointing  to  another,  etc.,  to  handle  complex  transformations. 
Access  control  information  may  be  included  in  the  descriptor  data  so 
the  descriptor  reference  may  provide  both  the  isolation  and  the 
complete  mediation  required  of  the  security  kernel. 

Referring  to  Figure  h,  each  subject  has  a descriptor  base  root  (DBF.) 
which  points  to  a table  of  memory  descriptor  segments  (BASES)  and  to  a 
table  of  I/O  descriptor  segments  (BAS2I) . Each  descriptor  car  be 
indirect,  i.e.  a pointer  to  another  descriptor  (BASE  D-=001)  , or 
direct,  i.e.  describing  an  object  (BASE  D=100) , The  descriptors  also 
carry  access  permission  information.  A direct  memory  segment 
descriptor  may  specify  a memory  location  directly  for  unpaged  segments 
or  may  describe  the  memory  location  through  the  use  of  a page  table 
which  allows  paged  segments- 


PRCTECTICN  dECHANISaS 


Generalized  Domains 


Prior  work  (references  22  and  23)  has  led  to  the  concept  of  a region 
or  "domain”  of  execution  tor  a process.  A domain  may  be  a partition 
of  the  process's  resources:  a subset  of  the  memory  and  peripheral 
device  space  available  to  the  process.  A set  of  access  control 
attributes  such  as  Read,  Write,  Execute  and  Call  may  be  associated 
with  each  domain.  Access  attributes  may  be  granted  or  removed 
simultaneously  for  all  elements  in  the  domain.  This  provides  a tool 
for  one  process  to  use  an  function  while  neither  granting  the  function 
access  to  more  information  than  it  needs  to  accomplish  the  requested 
service  nor  granting  the  subject  access  to  information  it  would  not 
otherwise  be  able  to  reach.  It  is  very  efficient  to  associate  a 
large,  arbitrarily  chosen,  set  of  segments  to  form  a common  domain. 
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DESCRIPTOR 

SEGMENT 


' Descriptor  Structure 


Substantial  research  is  in  progress  in  this  area,  including 
Schroeder's  irutually  suspicious  domains  (Reference  8),  Bedell's 
revocable  capabilities  (Reference  10),  the  capability  procagation 
ideas  of  the  Hydra  system  (Reference  11),  the  work  of  Needham  and  his 
associates  on  the  Cambridge  system  (Reference  12) , the  work  at  the 
Stanford  Research  Institute  (Reference  30) , etc.  A survey  of  this 
work  was  recently  published  by  Fabry  (Reference  13)  . 

The  general  domain  alternatives  were  considered  from  the  military 
security  viewpoint,  but  unsolved  problems,  were  found  in  all 
investigated.  Ho  hardware  yet  exists  to  readily  support  a general 
domain  approach. 

Schroeder's  mutually  suspicious  domains  have  problems  in  passing 
parameters  and  validating  the  legitirracy  of  their  access,  and  do  not 
address  the  needs  of  Input/Output  in  a definitive  manner.  Fedell's 
revocable  capabilities  improve  on  the  propagation  ideas  of  the 
Cambridge  and  Hydra  system’s,  but  the  I/O  problem  is  not  definitively 
addressed. 


.^.gltics  Rjlng  Structure 

The  best  known  and  proven  device  for  associating  an  arbitrarily  chosen 
set  of  seaments  into  a domain  is  the  protection  ring  and  bracket 
concept  introduced  in  the  Multics  system.  It  has  been  described  by 
Saltzer  and  Schroeder  (Reference  14),  and  by  Organick  (Reference  15). 
The  years  of  practical  use,  experience  and  exposure  with  rings  in 
Jiultics  greatly  reduces  the  risk  of  unexpected  performance  degradation 
and  certification  problems  compared  with  the  unproven  alternatives. 

Therefore  the  design  decision  was  made  to  use  a Multics-like 
descriptor  design  implementing  a ring  protection  structure.  The 
minimum  reguirement  for  a secure  system  is  two  rings,  one  for  the 
security  kernel  and  one  for  everything  else.  Prior  work  has  shown 
that  another  ring  is  useful  to  separate  system  programs  from  user 
programs,  so  the  reguirements  specified  at  least  three  domains.  Since 
this  reguires  two  bits  for  ring  description,  a four  ring  system  was 
selected.  Four  rings  provide  one  ring  for  the  security  kernel,  one 
ring  for  the  operating  system,  and  one  ring  for  application  programs. 
The  remaining  ring  may  be  used  for  the  operating  system,  as  this 
allows  partitioning  of  operating  system  functions.  Alternately,  the 
ring  might  be  used  to  provide  two  levels  for  application  programs. 

This  would  give  the  potential  for  design  of  protected  subsystems. 
Choice  of  use  (or  ignoring)  the  extra  ring  is  left  to  the  designers  of 
the  software  system.  The  design  selected  contains  unused  descriptor 
coding  options  to  permit  extension  if  it  is  deemed  necessary  within 
the  lifetime  of  the  SPK. 


TLe  hierarchical  rinq  structure,  as  illustrated  in  Figure  5,  considers 
the  doirains  as  concentric  barriers,  with  execution  in  the  innermost 
rings  the  most  privilege!.  Descriptors  used  for  controlling  mapping 
include  the  ring  privilege  informiation  of  the  subiect.  The  usages  of 
rings  1,  2,  and  3 shoun  in  this  figure  is  only  one  possible 
assignment.  Other  allocations  of  these  rings  are  possible. 

Multics  practice  has  shown  the  value  of  using  three  ring  boundary 
indicators,  called  "rinq  brackets".  An  ordered  triple  of  ring  numbers 
(F1,E2,?3)  specify  the  rings  in  which  Pead-and-Write  (0  < Heff  < B1), 
Read-and-5xacute  {?1  < Peff  < R2)  , and  Procedure  call  (31  < Reff  < E3) 
are  oermissible.  (5)  No  access  is  permitted  with  Reff  > S3.  A 
required  relationship  is  El  < B2  < R3.  Figure  6 shows  the 
interpretation  of  the  rinq  bracket  numbers. 

The  resource  descriptor  specifies  the  values  of  31,  32,  and  33  for  the 
ohiect.  S??i  hardware  maintains  a record  of  the  current  ring  of 
execution.  The  effective  rinq  of  execution  is  controlled  by  the  SFK 
and  can  only  he  changed  by  interrupt,  trap,  CALL,  FETUHN,  or  by  the 
security  kernel's  using  a privileged  instruction.  The  validation  of 
CALLS  and  their  arguments  and  the  handling  of  traps  and  interrupts  by 
the  security  kernel  ensures  that  users  cannot  charge  their  ring  of 
execution  tc  breach  security- 


I ; (5)  Reff  is  the  effective  rinq  number,  which  is  the  current  ring 

I < number  as  modified  by  the  CALL  mechanism.  See  Section  V,  "Ping 

Crossing". 
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Figure  6 
^ng  Brackets 


SECTION  III 


DESIGN  CONCEPT 


INTHODUCTICN 

This  section  presents  an  overview  of  the  design  of  the  system  to 
provide  a reference  monitor  and  a useful  Secure  Communications 
Processor  system.  First  the  concept  of  levels  of  abstraction  is 
introduced  and  then  the  system  is  described  using  these  terms. 
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LEVELS  CF  ABSTRACTION 
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outputs  of  the  design. 


Abstraction  is  a way  of  avoiding  complexity  and  a mental  tool  by  means 
of  which  a finite  piece  of  reasoning  can  cover  a myriad  of  cases 
(Reference  16).  The  purpose  of  abstracting  is  not  to  be  vague,  but  to 
create  a semantic  level  in  which  one  can  be  absolutely  precise. 
Diikstra's  levels  of  abstraction  have  been  demonstrated  to  be  a 
powerful  design  methodology  for  complex  systems,  most  notably 
Diikstra's  "THE”  system  (Reference  17)  and  the  Venus  Operating  System 
(Reference  18) . In  general,  the  use  of  levels  of  abstraction  leads  to 
a better  design  with  greater  clarity  and  fewer  errors.  A level  is 
defined  not  only  by  the  abstraction  that  it  supports  (for  example,  a 
segmented  virtual  memory)  but  also  by  the  resources  employed  to 
realize  that  abstraction.  Lower  levels  (closer  to  the  machine)  are 
not  aware  of  the  abstractions  or  resources  of  higher  levels;  higher 
levels  may  apply  the  resources  of  lower  levels  only  by  appealing  to 
the  functions  of  the  lower  levels.  This  pair  of  restrictions  reduces 
the  number  of  interactions  among  parts  of  a system  and  makes  them  more 
explicit. 

Each  level  of  abstraction  creates  a virtual  machine  environment. 
Programs  above  some  level  do  not  need  to  know  how  the  virtual  machine 
of  that  level  is  implemented.  For  example,  if  a level  of  abstraction 
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creates  sequential  processes  and  irultiplexes  one  or  more  hardware 
processors  amonq  them,  then  at  higher  levels  the  number  of  physical 
processors  in  the  system  is  not  important. 

3Y  the  rules  of  levels  of  abstraction,  calls  to  a procedure  at  a 
different  level  must  always  be  made  in  the  downward  direction,  and  the 
corresponding  return  in  the  upward  direction.  For  maximum,  clarity, 
.downward  calls  should  be  to  the  next  lower  level,  but  there  will 
always  be  cases  where  calls  that  skip  over  one  or  more  levels  can  be 
iustified.  Returns  are  always  to  the  calling  program,  except  in  the 
event  of  a severe  error  where  several  of  the  calling  procedures  may  be 
skipped  over  by  the  return.  Figure  7 shows  the  structure  of  a system 
where  most  calls  are  to  functions  of  the  next  lower  level,  but  the 
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Figure  7 .Levels  of  Abstraction 


level  1 function  x is  called  from  levels  2,  3,  and  4, 

When  a ring  (hierarchical  domain  of  execution)  structure  is  added  to 
the  system,  simplicity  is  enhanced  by  having  each  ring  consist  of 
contiguous  levels.  Thus  the  kernel,  which  must  be  the  innermost  ring 
(ring  0),  should  consist  of  the  level  of  abstraction  that  implements 
the  reference  monitor  concept  and  the  supporting  levels  beneath  that 
level.  In  our  example  system,  the  boundary  between  ring  0 and  ring  1 
may  come  between  level  2 and  level  3 as  shown  in  Figure  8.  Following 
the  policy  of  making  a ring  consist  of  contiguous  levels,  all  cross 
ring  calls  are  automatically  to  an  inner  ring.  This  type  of  ring 
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Flgiire  9 Outward  Ring  Crossing 


the  use  of  orotectiori  rings  causes  the  conflict.  A machine  that 
provided  the  more  general  form  of  non-hierarchical  protection  domains 
would  solve  this  problem  by  allowing  an  internal  partitioning  of  the 
ternel.  Domain  machines,  however,  are  not  currently  available.  Since 
we  have  decided  to  take  advantage  of  the  proven  theory  of  a ring 
machine  and  we  believe  that  the  levels  of  abstraction  design 
methodology  will  facilitate  certification  of  the  kernel,  we  must 
consider  compromising  one  or  both  of  the  design  goals  of  overall 
system  clarity  and  a small  simple  kernel.  This  issue  will  be 
discussed  further  as  design  details  are  presented. 


LEVELS  CF  THE  KERNEL 

111  designing  the  hardware  to  support  a security  kernel,  levels  of 
abstraction  have  been  used  in  the  translation  of  the  abstract  elements 
of  the  mathematical  model  to  tangible  elements  of  a secure  computer 
system.  The  first  steps  taken  were  to  make  an  interpretation  of  the 
model  elem.ents  (i.e.,  obiects  are  virtual  m.emory  segments  and  sub;Jects 
are  sequential  processes)  and  to  provide  at  some  level  of  abstraction 
a set  of  functions  that  controls  access  to  these  elements.  Thus  the 
abstraction  created  by  this  level  is  that  of  a secure  computer  system. 
It  must  be  emphasized  that  what  this  secure  system  level  of 
abstraction  does  is  to  effect  the  implementation  of  the  reference 
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iroritoi:,  thus  insuring  that  the  system  is  always  in  a secure  state 
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The  specific  design  structure  chosen  for  the  actual  iirplementation  of  • 
the  interpreted  elements  is  done  by  lower  levels  of  abstraction,  as 
shown  by  Figure  10.  While  the  software  at  these  levels  is  not 
cognizant  of  specific  security  reguirements,  it  is  part  of  the  kernel 
because  the  correct  operation  of  the  secure  system  level  functions 
depends  upon  the  correctness  of  lower  levels.  Ke  choose  to  place  the 
segmented  virtual  memory  level  above  the  process  level  because 
segments  can  be  shared  by  processes,  and  because  we  want  to  be  able  to 
star'*’  a new  process  running  when  the  current  process  must  wait  for  a 
segment  to  he  swapped  into  main  memory. 

It  should  be  clear  that  the  boundary  of  the  kernel  belongs  immediately 
above  the  secure  system  level  cf  abstraction.  Software  outside  of 
this  perimeter  can  execute  the  unprivileged  haraware  instructions  and 
invoke  the  functions  provided  by  the  secure  system  level  with 
arbitrary  arguments.  Since  the  unprivileged  machine  instructions 
cannot  put  the  system  into  a non-secure  state  and  the  secure  system 
functions  are  proven  secure  for  all  arguments  passed  to  them,  the 
security  of  the  system  is  independent  of  what  the  software  above  the 
secure  system  level  of  abstraction  does  or  does  not  do.  Thus  the 
implementation  of  the  security  level  of  abstraction  and  the 
implementation  cf  the  lower,  supporting  levels,  gives  us  a complete 
security  kernel. 

Five  levels  of  abstraction  ate  used  to  describe  the  system  including 
the  security  kernel.  The  highest  (Level  4)  includes  m.cst  of  the 
soft-ware  of  the  system  and  is  exterior  to  the  security  kernel.  The 
next  level  (Level  3)  holds  the  security  kernel  procedures  responsible 
for  checking  acceptability  of  access  to  resources  and  for  reguesting 
the  levels  belcw  to  create  appropriate  descriptions  of  the  resources. 
The  system  is  tc  be  implemented  so  that  no  resource  may  be  used 
without  this  description  provided  by  the  lower  levels.  Level  2 
creates  a segmented  virtual  m.emory  for  the  use  of  processes  running  at 
higher  levels-  Level  1 creates  and  manipulates  the  machine  language 
descriptions  of  the  resources  handled  by  the  higher  levels.  level  0 
is  the  hardware  of  the  system  itself.  Cur  concern  in  this  study  was 
to  design  the  hardware  of  Level  0 so  that  the  functions  of  the  higher 
levels  could  be  supported  completely  and  efficiently. 


LfVELS  4 AND  GREATER  - UNCERTIFIED  SOfTKAPE 

The  software  of  Level  4 (and  higher)  is  the  reason  for  existence  of 
the  entire  system.  The  applications  programs,  such  as  communications 
programs,  message  switching  software,  or  front-end  processing  programs 
operates  at  the  higher  levels.  At  Level  4 is  the  operating  system 


which,  in  turn,  exists  to  serve  and  support  the  applications  programs. 
The  operating  system  at  level  4 can  serve  to  mask  the  operation  of  the 
security  kernel  from  the  applications  software  at  the  higher  levels, 
thereby  facilitating  the  transfer  of  applications  from  a non-secure  to 
a secure  environment. 

Security  rules  and  limits  are  visible  at  Level  4 principally  as  bounds 
on  permissible  operations.  An  application  program  does  not 
necessarily  have  to  be  aware  of  its  own  clearance  or  of  the 
classification  of  the  obiects  it  manipulates.  It  is  essentially 
encaosulated  in  a virtual  machine  whose  extent  is  specified  by  the 
clearance  of  the  user  initiating  the  application  program.  Any  attempt 
to  escape  the  encapsulation  results  in  an  automatic  call  to  a more 
privileged  level.  The  application  program  that  attempts  to  exceed  its 
limits  may  be  suspended  or  terminated  without  notification  to  the 
program  itself. 


LEVEL  3 - SECUFITY 

The  software  of  level  3 sees  a virtual  machine  with  a segmented 
virtual  memory  that  provides  for  access  control  and  the 
multiprogramming  of  sequential  processes.  Thus,  the  major  elements  of 
the  mathemiatical  model  of  secure  computer  systems  (subjects,  objects 
and  access  controls  are  realized.  Given  the  environment  assumed  by 
the  model,  the  implem.entation  of  security  by  following  the  rules  of 
the  model  is  straightforward. 

Level  3 has  no  hardware  resources  of  its  own.  Level  3 makes,  based  on 
the  rules  of  the  model,  a correspondence  between  the  subjects  and 
objects  of  the  model  and  the  abstractions  implemented  by  levels  1 and 
2,  associates  security  attributes  with  these  lower  level  abstractions, 
and  controls  access  to  the  lower  level  functions  that  operate  on  these 
abstractions.  Each  model  rule  has  two  parts  - the  first  part  consists 
of  security  checks  to  determine  if  the  reguested  state  change  can  be 
permitted;  the  second  part  of  the  rule  indicates  how  the  state  change 
is  to  be  made.  In  the  kernel,  level  3 functions  perform  security 
checking  and  then  direct  levels  1 and  2 to  perform  state  changes  if 
security  requirements  are  satisfied.  Level  3 maintains  the  access 
matrix  which  records  the  security  level  information  on  all  subjects 
and  obiects.  This  is  used  to  direct  the  lower  level  machines  to 
invoke  the  proper  access  control. 

The  kernel  uses  processes  as  the  basic  interpretation  of  subjects,  and 
segments  as  the  basic  interpretation  of  objects.  In  addition, 
semaphores  and  interprocess  communication  messages  are  also  objects. 
The  data  structures  used  by  level  1 to  support  processes  include  a 
specification  of  each  process's  current  address  space  (the  segments 


accessible)  and  an  identification  of  the  user  associated  with  the 
process  together  with  the  user's  security  attributes. 

t 

Since  the  model  has  been  iirpletrented  at  Level  3,  all  remaininq 
software  above  it  in  the  system  can  be  uncertified  - contain  buqs  or 
malicious  penetration  attempts  - without  a threat  of  security 
compromise  if  two  conditions  are  satisfied.  First  the  kernel  must  be 
protected,  and  second,  access  to  its  functions  must  be  controlled. 
These  conditions  are  tret  by  preventinq  uncertified  software  from 
qaininq  write  access  to  the  kernel  seqments  and  by  permittinq  only  the 
kernel  to  execute  in  the  inner  rinq. 


LEVEL  2 - SEGMENTED  VIFIOAL  MEMORY 


The  second  level  of  abstraction  creates  a seqm.ented  virtual  memory, 
buildinq  on  the  seqmented  main  memory  provided  by  level  0 (the 
hardware).  Seqments  are  the  primary  storaqe  entities  of  the  system 
and  will  be  the  basic  obiect  to  which  access  is  controlled  by  the 
security  level  of  abstraction.  Since  even  the  largest  hardware 
segment  may  be  small  for  some  apalications , we  anticipate  the  creation 
of  a virtual  memory  abstraction  . This  abstraction  will  allow  several 
seqments  to  be  treated  as  a single  obiect  and  permit  subsections 
(seqments)  to  be  individually  swapped  in  and  out  of  main  memory. 

Level  2 implements  a one  level  virtual  memory  because:  1)  segments 
are  the  only  type  of  storaqe  entity,  and  2)  as  different  segment 
descriptors  are  loaded  into  the  segmentation  registers  the  address 
space  of  a process  can  be  greater  than  the  size  of  main  memory.  For 
applications  requiring  very  large  collections  of  data  the  operating 
system  can  create  a file  structure  at  level  U or  higher  using  the 
virtual  memory  created  here. 


One  would  like  (as  Multics  does)  to  implement  variable  sized  segments 
consisting  of  fixed  sized  pages-  The  use  of  paging  facilitates  the 
dynamic  growth  of  segments,  permits  only  part  of  a segm^ent  to  be 
swapped  into  main  memory,  and  vastly  simplifies  the  allocation  of  both 
primary  and  secondary  memory.  This  has  been  left  as  an  option  for 
implementation. 


Segments  have  attributes  - information  that  describes  the 
characteristics  of  a segment.  From  the  mathematical  model  we  know 
that  at  level  3 seqments  will  have  security  attributes;  at  level  2 
they  have  implementation  attributes.  Implementation  attributes 
include  a segment's  size  and  disk  address.  The  attributes  of  a 
segment  are  contained  in  an  entry  in  a directory.  A directory  is  an 
obiect  which  describes  other  obiects.  Each  directory  entry  contains 
the  attributes  of  the  named  obiect.  At  level  2 space  is  provided  in 
directory  entries  for  security  attributes  but  the  operation  of  this 
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level  is  independent  of  the  values  of  security  attributes.  To 
simplify  conceptual  design  and  iirpleirentation,  directories  are 
themselves  segments  with  attributes  residing  in  other  directories,  the 
total  structure  is  a directory  hierarchy  in  the  form  of  a tree.  The 
attributes  of  the  root  segment  (directory)  of  this  tree  are  fixed  by 
the  design  and  implementation. 

All  segments  in  the  hierarchy  are  either  directory  segm.ents  or  data 
segments.  (Segments  containing  executable  code  are  considered  data 
segments  by  level  2.)  Although  level  2 does  not  enforce  access 
control  to  segments  in  general,  it  cannot  permit  software  above  it  to 
write  directly  into  directory  segm.ents,  because  the  correct  operation 
of  level  2 reguires  the  integrity  of  the  (implementation)  attributes 
of  segments.  functions  at  this  level  provide  users  with  an 
interoretiva  directory  write  capability.  The  security  reguirements 
enforced  at  the  security  level  will  further  restrict  access  to 
directories  because  of  the  nature  of  some  of  the  segment  attributes. 

To  allow  dynamic  segment  sharing,  level  2 associates  a semaphore  (5) 
with  each  segment  and  reguires  write  access  to  the  segment  in  order  to 
operate  on  the  semaphore.  (See  the  discussion  of  Level  1 for  more 
detail  on  semaphores  and  the  operations  on  them) . The  I/O  segment 
semaphores  have  a special  use  - the  )cernel  translates  I/O  interrupts 
into  releases  of  the  appropriate  semaphores.  Thus,  when  a process 
wishes  to  wait  for  an  interrupt  from  an  I/O  device,  it  seizes  the  I/C 
segment  semaphore,  (presumably)  blocking  itself.  When  the  interrupt 
occurs,  a release  is  performed  and  the  process  becomes  unblocked.  The 
kernel  is  only  concerned  with  controlling  access  to  the  I/O  segments 
and  semaphores,  not  with  the  correct  use  necessary  to  assure  proper 
synchronization. 

Level  2 implements  a segmented  virtual  memory  by  building  upon  level 
O's  segmented  main  memory,  using  secondary  storage  devices  for  segment 
swapping,  and  employing  a data  base  to  indicate  the  state  of  the 
virtual  memorv.  The  data  base  consists  of  the  directory  segments, 
tables  for  managing  the  allocation  of  secondary  storage,  and  the 
Active  Segment  Table  (AST) . The  AST  is  a mechanism  that  facilitates 


(6)  A semaphore  is  a special  variable  accessible  only  to  two  special 
operations,  seize  (privi,  ?)  and  release  (vrij,  V).  When  a process 
reeds  exclusive  access  to  a resource,  it  must  seize  (P)  the  associated 
semaphore.  If  the  resource  is  in  use  when  the  seize  operation  is 
invoked,  the  process  is  suspended  until  some  future  time  when  the 
resource  is  released  by  another  process.  After  a process  is  finished 
with  exclusive  access  to  a resource,  it  must  invoke  the  release  (V) 
operation  to  allow  other  processes  access  to  the  resource.  See 
Peterence  31. 
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ths  sharim  of  seqrrents  ir  irain  ir.eirory  - if  two  different  processes 
wish  to  access  the  saire  saqirent  they  both  access  the  same  physical 
noairent  and  not  two  different  copies.  Any  segment  that  is  in  the 
address  space  of  one  or  more  processes  or  is  "wired  down"  (permanently 
swapped  into  main  memory)  is  active  - it  has  an  entry  in  the  AST.  An 
active  seqinent  table  entry  (ASTE)  contains  the  segment's  permanent 
attribui-es  - copied  from  the  directory  - as  well  as  additional 
attributes  associated  with  the  fact  that  the  segment  is  active.  These 
additional  attributes  include  a list  of  the  processes  that  have  the 
segment  in  their  address  space  and  the  main  memory  address  of  the 
seqinent  if  it  is  currently  swapped  .in. 

Segments  in  the  hierarchy  can  be  uniquely  identified  in  a variety  of 
way.s.  If  a segment  is  active,  identifying  its  entry  number  in  the  AST 
(fST2#)  specifies  the  segment.  If  a seqinent  is  not  active  but  its 
parent  is,  then  the  ASTE#  of  the  parent  directory  and  the 
identification  of  the  entry  within  the  directory  that  contains  the 
seqment's  attributes  (directory,  entry#)  specifies  the  segment.  A 
generalization  of  the  (directory,  entry#)  identification  method  is  the 
comolete  pathname  - a specification  of  all  directory  entries, 
beginning  with  the  root,  that  identify  the  segment.  Finally,  each 
segment  has  a unique  igentifier  - its  disK  address.  Within  the 
security  kernel  the  primary  segment  identification  techniques  are  the 
•ISIE#  and  the  (directory,  entry#).  See  Reference  23. 

If  provision  is  made  for  descriptors  of  I/O  and  peripheral  devices, 
which  are  also  forced  to  be  completely  unalterable  by  any  higher 
numbered  level.  Level  2 supports  all  system  storage  in  a secure 
manner.  That  is,  I/O  and  peripheral  devices  may  be  accounted  for  by 
data  structures  analoaous  to  those  for  segments.  If  compatible 
controls  ate  placed  on  I/O  and  peripheral  devices,  they  can  be  handled 
like  segments. 

ft 

Level  2 provides  functions  for  creating  and  deleting  segments,  adding 
and  removing  segments  from  a process's  address  space,  and  creating  and 
destroying  segment  descriptors,  for  both  data  and  directory  segments. 
The  segments  created  at  this  level  are  the  basic  interpretation  of  the 
obiects  of  the  nathematical  model.  Although  segment  descriptors 
permit  access  ccntrol  to  segments,  the  only  access  control  policy 
enforced  at  this  level  is  the  reguirement  for  interpretive  directory 
writes. 


LEVEL  1 - SECUENTIAL  FRCCESSES 


Level  1 creates  the  process  abstraction, 
somewhat  vague)  definition  of  process  - a 
execution.  See  Reference  23.  The  design 
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We  use  the  "standard"  (and 
process  is  a procedure  in 
supports  a variable  number 


of  processes;  each  runs  on  a virtual  machine  and  consists  of  an 
address  space  and  control  information  about  the  process.  At  level  1 
it  is  sufficient  to  know  that  the  address  space  is  defined  by  the 
control  information,  part  of  which  is  the  contents  of  the  descriptor 
segment.  Level  1 software  has  the  responsibility  for  allocating  the 
processor  to  one  of  the  processes  whose  dynamic  progress  is 
permissible. 

At  a given  time,  a process  is  in  one  of  several  possible  execution 
states  (Reference  19).  Figure  11  shows  the  relationships  among  the 
various  execution  states  and  the  actions  that  move  a process  from  one 
state  to  another.  In  the  inactive  state  a process  does  not  have  an 
address  space  ard  cannot  run.  A process  can  only  be  moved  out  of  (and 
also  into)  the  inactive  state  by  the  supervisor,  a special  executive 
process  that  is  never  in  the  inactive  state.  At  the  time  that  it 
moves  a process  out  of  the  inactive  state,  the  supervisor  must 
establish  the  initial  address  space  of  the  process.  The  purpose  of 
the  inactive  state  is  to  create  a mechanism  for  minimizing  the 
resources  reauired  to  support  a process  that  is  not  currently  needed. 

An  active  process  is  either  blocked  or  unblocked-  In  the  blocked 
state,  a process  is  waiting  for  the  occurrence  of  som.e  event.  An 
urblocked  process  is  either  in  the  running  state  or  the  ready  state. 
The  running  state  simply  signifies  that  the  process  has  the  CPO 
allocated  to  it.  Above  level  1 the  running  and  ready  states  are 
logically  eguivalent.  In  the  ready  state  a process  is  ready  to  run 
(its  dynamic  progress  is  permissible)  but  must  wait  for  the  CPU  to  be 
allocated  to  it.  Processes  enter  the  ready  state  from  the  blocked 
state  when  the  event  for  which  they  were  waiting  occurs.  Transition 
of  processes  between  the  ready  and  running  states  is  controlled  by  a 
mechanism  internal  to  level  1. 

The  hardware  resources  of  this  level  are  the  CPU  and  a real  time 
clock.  A data  base  is  employed  to  contain  state  information  about  the 
processes  and  to  help  manage  them.  This  state  information  includes  a 
definition  of  each  process's  address  space,  an  indication  of  its 
execution  state,  and  a specification  of  the  user  associated  with  the 
process  and  his  security  attributes.  No  interpretation  of  these 
security  attributes  is  made  at  this  level  (the  operation  of  level  1 is 
independent  of  their  value).  Rather,  space  is  set  aside  in  level  1's 
data  base  for  security  attributes  as  a convenience  for  higher  levels. 

Several  different  types  of  functions  are  provided  by  level  1.  Two 
sets  of  functions  are  provided  for  the  synchronization  of  processes  - 
semaphores  and  Diikstra's  P and  V functions  primarily  for  intraprocess 
coordination  along  with  message  send  and  receive  for  interprocess 
communication.  In  the  level  2 subsection  we  noted  how  P and  V are 
used  to  handle  I/C  interrupts. 
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Iev9l  1 iirpleirents  a siirple  schadulinq  mechanism  - the  highest 
priority  (7)  process  that  is  ready  to  run  has  the  CPU  allocated  to  it. 
To  allow  a more  sophisticated  scheduling  policy  to  be  implemented 
outside  of  the  kernel  a function  can  be  provided  to  dynamically  change 
process  priorities.  The  uncertified  operating  system  (Level  4) 
normally  provides  this  scheduling  policy. 

Finally,  as  online  users  log  on  and  off  the  system,  communications 
lines  or  control  procedures  become  active  or  inactive,  and  as  batch 
iobs  are  initiated  and  terminated,  it  is  necessary  to  provide  them 
with  processes  and  then  to  terminate  these  processes.  Two  functions  - 
activate  process  and  deactivate  process  - are  provided  for  this 
purpose. 

Level  1 creates  a multiprogramming  environment  which  effectively 
implements  the  coexisting  subjects  that  are  a major  element  of  the 
model. 


LEVEL  0 - TUE  HAPCSAHE 

The  abstraction  at  Level  0 is  that  of  a (possibly  multi-)  processor 
with  augmented  main  memory  and  a set  of  peripheral  devices.  The  basic 
resources  used  in  realizing  this  abstraction  are  the  hardware  of  the 
protected  system  (CPU,  memory,  peripheral  controllers,  peripheral 
devices)  and  the  SEM.  This  hardware  is  extended  by  software  in  the 
security  kernel  (operating  in  Ring  0)  and  perhaps  by  firmware  in  some 
of  the  peripheral  controllers. 

The  CPU  of  the  protected  system  provides  arithmetic  and  logical 
functions  to  support  the  process  abstraction,  which  is  used  both  for 
the  kernel  and  for  the  applications  programs.  The  CPU  also  supplies 
the  real  time  clock  function,  perhaps  augmented  by  facilities  of  the 
S?M  to  give  unique  identifiers  for  processes.  The  CPU  and  the  SPM 
work  together  to  support  the  augmented  main  memory  which  provides  for 
the  segmented  virtual  memory  abstraction.  The  CPU  and  the  SPM  also 
coooerate  in  providing  the  interrupt/trap  facility  and  the  ability  to 
oerform  secure  fast  context  switching.  The  SPM  provides  hardware 
supaort  for  access  control,  procedure  segregation,  protection  rings, 
and  seqman*-  description,  as  well  as  providing  for  synchronization  of 
multiole  processors.  Descriptor  usage  is  enforced  both  for  the  rapid 
core  memory  accesses  and  for  the  slower  peripheral  device  accesses. 


(7)  Priority  as  used  here  may  consist  of  many  variables  as  perceived 
by  the  overall  system.  At  this  level  however,  all  considerations  are 
reduced  to  a single  integer  used  for  ordering. 
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SECTION  IV 


SPH  DESIGN 


INTRO DUCTION 

This  section  will  be  devoted  to  an  expansion  of  the  requirements  and 
characteristics  of  the  hardware  needed  to  produce  a Secure 
Communications  Processor.  Particular  attention  will  be  paid  to  the 
specification  of  functions  for  the  new  element  in  the  system,  the  SPM. 
Software  requirements  will  be  mentioned  only  when  needed  to  identify  a 
hardware  capability  or  when  the  function  is  deemed  best  carried  out  by 
software. 

The  reader  will  recall  the  picture  of  the  ideal  reference  monitor, 
shown  in-  ?iqure  1.  The  idealized  reference  monitor  sits  between  all 
subiects  and  all  obiects  and  mediates  every  reference  cf  a subject  to 
an  object.  The  SFH  has  been  devised  to  follow  this  ideal  as  closely 
as  practical.  The  central  position  of  the  SPM  was  shown  in  the 
protected  computer  system  of  Piqure  2. 

The  blocic  diaqram  of  Fiqure  12  shows  a minicomputer  system  which  is 
suitable  both  as  a communications  processor  and  also  as  the  basis  for 
a orotected  system  and  thus  for  a Secure  Communications  Processor. 

Ml  components  of  the  system  are  connected  by  the  common  bus  which 
transfers  both  control  information  and  data  between  the  units  of  the 
svstem. 

?iqure  13  shows  the  same  minicomputer  system  with  the  addition  of  the 
SPH.  The  common  bus  is  now  divided  into  a virtual  bus  and  an  absolute 
PUS.  The  virtual  bus  carries  addresses  in  virtual  (unmediated)  form 
while  the  absolute  bus  cajpries  addresses  in  absolute  (mediated)  form.. 
The  SPM  serves  to  mediate  anc  translate  the  addresses  from  virtual  to 
absolute  form,  applyinq  the  rules  of  the  security  model  in  the 
process.  Tn  the  ideal  case,  the  SPH  would  perform  all  of  the 
functions  of  the  reference  monitor  and  realize  Picture  1 directly. 
Unfortunately  this  would  reauire  that  the  SPM  essentially  duplicate 
the  hardware  of  the  system  to  be' protected,  losing  the  savings  that 
were  the  reason  for  introduction  of  the  SPM  in  the  first  place. 
Practical  considerations  force  us  to  make  multiple  use  of  the  hardware 
of  the  protected  system  to  implement  the  reference  monitor. 

In  figure  13,  ♦•he  device  coupler,  supporting  the  high  data  rate 
peripherals  - tape  and  disk  - is  placed  on  the  absolute  bus  in 
contravention  of  the  model  requirements.  This  is  a practical 
consideration,  since  every  use  of  the  virtual  bus  is  individually 
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naediated  b/  the  SFN,  the  addition  of  several  high  speed  peripherals 
transferring  data  at  the  same  time  could  drive  the  speed  reguirements 
of  the  SPM  beyond  the  current  state' of  the  art.  The  device  coupler 
being  on  the  absolute  bus  must  assume  some  of  the  duties  of  the 
reference  monitor  in  I/O  transfers. 

Hardware  of  the  SF!1,  CPU,  and  memory  components  of  the  system  is  used 
tc  support  the  security  kernel.  The  SPM  provides  required  security 
capabilities  rot  built  into  the  CPU,  acting  as  a ''security  unit"  in 
much  the  same  fashion  as  a "floating  point"  unit  supplies  its 
services.  The  CPU  provides  logic  and  arithmetic  capabilities  for  the 
various  processes,  among  them  the  security  kernel,  operating  system, 
and  application  programs.  The  mem.ory  stores  instructions  and  data  for 
all  processes  from  security  kernel  through  application  program. 

The  security  kernel  operates  as  normal  software,  using  the  CPU  and 
memory.  The  security  kernel  operates  in  Ring  0,  which  grants  it  full 
system  privileces.  Among  these  privileges  is  the  ability  to  directly 
address  and  control  the  3EM,  the  ability  to  address  and  modify  the 
descriptors  stored  in  the  memory,  and  rhe  ability  to  utilize 
restricted  instructions,  functions  cf  the  SPM,  and  other  resources 
which  are  forbidden  to  other  (non-Ring  0)  software. 

The  operating  system  is  also  software  which  uses  the  CPU  and  memory. 

It  operates  in  Ping  1 (and  possibly  in  Ring  2 as  well).  The  operating 
system  has  some  of  the  privileges  normally  associated  with  a "master 
mode"  (I/O  functions) , but  none  of  the  special  privileges  accorded  to 
the  security  kernel  in  Ring  0.  The  operating  system  may  call  on  the 
security  kernel  for  sensitive  activities  essentially  in  the  same 
fashion  that  an  application  program  calls  on  the  operating  system. 

The  action  of  the  SPM  in  trarslating  and  checking  addresses  is 
invisible  to  the  operating  system.  The  Secure  Communications 
Processor  will  require  design  and  implementation  of  an  operating 
system  to  properly  couple  with  the  security  protection  characteristics 
of  this  system. 

Application  programs  also  operate  using  the  CPU  and  memory,  but  reside 
in  the  outer  ring  (or  rings).  They  have  the  restricted  privileges 
normally  associated  with  a "slave  mode",  calling  on  the  operating 
system  for  services  which  are  beyond  their  realm  of  privilege.  The 
actions  of  the  SPM  are  invisible  to  the  applications  programs,  except 
for  security  constraints.  It  is  expected  that  applications  or  systems 
crocrams  that  operate  on  the  unprotected  system  will  continue  to 
operate  on  the  protected  system  of  the  Secure  Communications- 
Processor,  so  long  as  they  obey  the  rules  of  security.  Any  attempt  to 
communicate  beyond  their  security  privilege  will  result  in  suspension 
of  the  offending  process. 
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The  choice  of  four  cinqs  for  the  SPM  leaves  a major  system  design  area 
open  for  the  iirpleirentor  of  the  secure  system.  There  is  one  ring 
available  for  use  to  partition  the  operating  system  or  to  partition 
the  applicatiur  programs  or  to  partition  the  kernel  if  it  proves  to 
aid  verif icat ior.  If  the  designer  chooses  to  use  it  for  the  operating 
system,  then  the  operating  system  will  occupy  2ing  1 and  5inq  2.  This 
choice  allows  the  operating  system  to  be  divided  into  sections  of 
oreater  and  lesser  privileae  and  may  lead  to  a cleaner  and  simpler 
design  than  is  rorirally  encountered.  Alternately,  the  designer  may 
chose  to  use  tvo  rings  for  application  programs.  This  will  allow 
definition  of  protected  subsystems  by  users  of  the  computer. 
Application  programs  could  then  be  partitioned  to  use  the  two  rings 
much  as  a conventional  computet  system  uses  its  "master"  and  "slave" 
modes.  The  restrictions  placed  on  the  designer  are  use  of  Ring  0 for 
the  security  kernel,  use  of  Ring  1 for  the  operating  system,  and  use 
of  the  two  remaining  outer  rings  as  best  suits  the  particular 
situation. 


THE  SPM 

Figure  14  shows  a block  diagram  of  the  SPM,  Within  the  SPM  there  is  a 
set  of  control  registers  and  logic  circuits  which  perform  the  security 
functions  and  respond  to  the  SPM  corrmands  issued  by  the  security 
kernel.  At  this  level,  the  security  kernel  is  recognized  as  the  CFO 
operating  in  Einq  0.  There  is  also  a mechanism  that  performs  the 
address  mapping  from  virtual  address  to  absolute  address  and  also 
checks  for  proper  access  permission.  If  an  error,  or  an  attempt  to 
violate  the  security  rules  is  detected,  then  the  security  kernel  is 
signaled  via  the  Error  Trap  Line.  This  trap  signal  must  bo  directed 
to  activate  the  security  kernel  (operating  in  Sing  0)  and  only  to  that 
orccess. 

Among  the  security  related  items  of  information  that  must  be  under  the 
control  of  the  SPM  hardware  are  the  current  Process_id,  the  current 
effective  ring  number,  and  the  descriptor  base  root  and  set  of 
descriptors  used  by  the  current  process.  These  items  may  be  stored  in 
the  memory  of  the  system,  rather  than  physically  within  the  SPM,  but 
they  must  be  under  the  exclusive  control  of  the  SPM  and  the  security 
kernel. 


SUPPORT  OF  THE  PROCESS  ABSTRACTION 

A process  was  referred  to  earlier  as  an  address  spaca/processor  state 
pair,  recognizing  that  a process  may  be  identified  both  by  the 
resources  it  owns  and  by  the  activity  of  the  processor  that  executes 
it.  It  is  essential  to  be  able  to  explicitly  identify  each  process 
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and  the  resources  allocated  to  it.  Some  form  of  process  identifier, 
referred  to  here  as  a process_id,  is  needed  to  bind  the  resources  and 
access  privileges  of  the  process  to  the  resources  allocated  to  the 
process. 

A unique  process_id  which  will  occur  only  once  in  the  lifetime  of  the  ^ 
system  seems  to  have  manv  advantaqes.  Such  an  id  could  be  generated 
for  a process  or  any  other  segment  by  a yery  large  (8) 
system  clock  or  by  combining  the  output  of  a counter  with  tbje''reading 
of  an  ordinary  date  and  time  clock,  or  by  simiply  using  a special 
”process„id  counter"  to  generate  each  id  as  required.  T>e  alternative 
of  using  and  reusing  process_id*s  from  a small  pool  may  have  virtues 
in  some  system  designs.  Algorithms  exist  to  guarantee  consistent  and 
secure  reassignment.  However,  the  final  form;  and  format  of  the 
process_id  is  left  to  the  software  system  designer. 

Several  other  items  of  inform:ation  serve  to  specify  the  process  and 
must  be  saved  and  restored  when  the  process  is  interrupted  and 
resumed.  These  items  include  the  Descriptor  Base  Root,  the 
descriptors,  the  effective  ring  number,  and  the  norm.al  data,  address, 
and  control  register  contents  of  the  CPO.  This  set  of  information 
will  be  referred  to  later  as  the  "context"  of  the  process.  In  this 
sense,  the  context  cf  a process  is  the  set  of  information  which 
defines  and  specifies  the  address  space/processor-state  pair  for  the 
process.  (9) 

It  is  particularly  vital  that  a communications  processoi^  be  arbX^  to 
switch  context  from  one  process  to  another  with  a miniff.um  of  system 
overhead.  The  communications  environment  typically  places  an 
extremely  heavy  lead  of  such  context  switches  on  the  system. 

Experience  with  the  security  kernel  on  the  PDP-11/45  indicates  that 
the  system  overhead  involved  here  may  be  the  principal  source  of 
pejrform^c^  degradation.  Thus  it  is  essential  that  the  CPU  selected 
support  a rapid  context  switch  and  that  the  SPH  be  designed  to 
facilitate  context  switches. 

The  switch  must  not  only  te  rapid,  but  it.  must  also  be  secure.  This 
means  that  special  care  must  be  taken  to  ensure  that  security  related 
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! (8)  Large  here  means  that  the  time  must  change  between  successive 

I hardware  reads  and  be  of  large  enough  extent  that  it  does  not  repeat 

I any  value  during  the  lifetime  of  the  system. 

I 

(9)  Note  that  many  processors  refer  to  a smaller  amount  of  information 


itj&irs"ot:  the  context  are  not  alterable  by  any  software  other  than  the 
security  kernel  itself. 

In  some  systems,  it  must  be  noted  that  the  protected  processor  itself 
may  be  wired  to  produce  context  switches  upon  occurrence  of  certain 
events,  usually  referred  to  as  "faults”  and  "interrupts".  "Faults" 
are  usually  taken  to  be  events  detected  automatically  within  the 
process  currently  beinq  executed  by  the  processor;  "interrupts"  ace 
events  detected  by  the  system  hardware  which  involve  actions  not 
necessarily  within  the  process  currently  in  execution  by  the 
processor.  Often  the  response  to  "interrupts"  may,  or  even  must,  be 
postponed  for  many  instructions,  but  a response  to  a fault,  especially 
if  it  indicates  a potential  hardware  failure,  usually  brooks  no  delay 
beyond  the  completion  - as  best  as  possible  - of  the  current  processor 
action.  For  this  reason,  the  protected  processor  miay  contain  within 
itself  implementation  of  automatic  partial  context  switchinq  in  these 
cases. 

A more  normal  implementation  of  fault  and  interrupt  processing  is  to 
switch  instruction  streams  and,  usually,  protection  rings,  to  process 
the  fault  or  interrupt.  The  result  of  this  processing  which  occurs 
within  the  process  in  operation  when  the  external  event  happened,  may 
he  a change  of  process.  However,  the  change  of  process  as  contrasted 
to  the  charge  of  instruction  stream  will  not  automatically  happen,  but 
will  be  the  result  of  an  analysis  of  the  event. 

The  system  must  be  rendered  secure  without  forcing  a gross  redesign  of 
these  functions.  The  presence  of  the  SPH  will  inevitably  introduce 
the  need  for  additional  security  related  and  non*secarity  related 
context  switchinq  information  to  be  generated  in  the  presence  of  both 
faults  and  interrupts.  The  SPM  design  must  add  this  information 
gracefully  and  securely  to  the  context,  to  resume  activity  later. 

The  security  kernel  must  also  be  the  controlling  entity  in  the 
creation  of  a new  process,  in  the  allocation  of  its  resources,  and  in 
the  eventual  release  of  those  resources  and  the  destruction  of  the 
process.  All  functions  involved  in  these  activities  must  have  the 
privilege  of  ♦he  security  kernel  (in  Ring  0).  The  functions  required 
of  the  SPM  will  be  listed  later. 


SUPPORT  0?  THE  AUGMENTED  MAIN  MEMORY  ASSTRACTIOH 

The  augmented  main  memory  abstraction  is  used  to  provide  the  virtual 
memory  of  higher  levels.  It  is  supplied  by  combining  use  of  virtual 
addresses  in  the  CPU  and  translation  of  these  addresses  to  absolute 
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forir  by  the  SFM.  The  concept  of  descriptors  and  of  descriptor  based 
addressing  «as  introduced  earlier.  Figures  3 and  4 introduced  the 
idea  of  address  translation  and  of  the  descriptor  technique. 

Use  of  descriptor  based  addressing  allows  the  choice  of  paged  or 
unpaged  menory  to  be  left  to  the  system  designer  and  brings  great 
consistency  to  all  memory  transformation  actions.  It  also  allows 
irclusior  of  access  checking  into  the  process  of  address 
transformation. 

Support  of  virtual  memory  requires  that,  in  the  address  transformation 
process,  a non-resident  segment  fault  (and  if  paging  is  used,  a 
non-resident  page  fault)  be  provided.  Since  in  an  active  paging 
system  supporting' several  processes  with  large  virtual  memories,  the 
page  fault  will  be  a very  common  occurrence.  The  efficient  support  by 
the  processor  and  the  3PM  of  page  and  segment  faults  is  another  area 
of  concern  in  architecture  selection.  If  paging  is  used,  the  page 
fault  processing  and  restart  must  be  efficiently  handled. 

Consideration  of  typical  minicomputers  used  for  communications 
processing  led  to  selection  of  a 22  bit  virtual  address  format  for 
illustrative  purposes.  This  format  is  shown  in  Figure  15,  set  up  for 
use  in  both  an  a paged  and  an  unpaged  environment.  The  selection  of 
22  bits  is  a compromise  between  the  desire  for  the  largest  possible 
virtual  memory  space  and  the  hardware  needed  to  support  the  larger 
virtual  address  field.  The  larger  address  field  imposes  a penalty  in 
the  CPU  where  it  is  necessary  to  manipulate  the  larger  address,  in  the 
memory  where  it  is  necessary  to  store  virtual  addresses  and  in  the  SFM 
where  it  is  necessary  to  mediate  them.  The  wider  hardware  paths  for 
virtual  addresses  cost  both  in  money  and  in  reliability  from  increased 
parts  count.  Typical  field  sizes  for  segment  number,  page  number  and 
offsets  have  been  selected  for  illustration,  but  the  actual  field  size 
used  is  at  the  discretion  of  the  system  designer.  Within  the  22  bit 
virtual  address  this  virtual  address  format  will  be  the  one  used  by 
the  CPU  and  transmitted  to  the  SFM. 

Figure  If.  shows  a typical  layout  for  the  Descriptor  Base  Root,  k two 
word  descriptor  base  is  provided  for  memory,  BASEM  and  BASES  Ext,  and 
another  for  Input/Cutput,  EASEI  and  BASEI  Ext.  Both  provide  for  a 
descriptor  type  field  "D”  which  may  be  used  to  indicate  chaining  or 
indirection  in  the  use  of  the  descriptors.  Both  also  provide  for  a 
limit  field,  used  to  restrict  the  extent  of  the  reference  permissible 
by  the  descriptor  base.  The  fields  marked  ”RFU"  are  reserved  for 
future  use. 

Fiaure  17  shows  typical  layouts  for  memory  descriptors  and  for  I/O 
descriptors.  Again,  the  field  lengths  are  typical  values  assuming  a 
16  bit  CPU,  subiect  to  alteration  by  the  system  designer.  Provision 
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is  made  for  a fault  director  field  "F"  for  use  by  the  security  kernel, 
for  three  rinq  tracket  numbers  (al,R2,R3),  for  three  access 
authorization  values  (Read,  Write,  Execute  - a,H,E)-  The  memory 
descriptor  also  provides  for  used  "U"  and  modified  "M"  bits  which  are 
set  "on"  by  the  SPH  when  the  memory  area  is  used  or  modified,  and  for 
a descriptor  type  field  "D"  which  specifies  direct  or  indirect 
descriptor.  Multics  experience  has  shown  the  value  of  these  bits  in 
selectinq  paqes  for  replacement  in  response  to  a page  fault.  A lock 
bit  "L"  and  a cache  bit  "C"  are  also  provided.  The  lock  bit  is  used 
to  lock  the  memory  area  and  is  set  and  reset  by  the  security  kernel 
software.  The  cache  bit  is  used  to  signal  the  suitability 
("encacheability")  of  the  descriptor,  or  the  data  it  describes,  for 
inclusion  in  a system  cache  memory  (if  one  is  used) . The  field  WCT  is 
used  as  a count  of  the  number  of  users  of  the  resource.  Experience  in 
system  design  indicates  that  provision  for  future  growth  should  be 
made.  The  base  address  (BASE)  and  its  extension  (BASE  EXT)  and  size 
limit  (LIMIT)  fields  are  provided. 


SriFTCHT  FOR  ACCESS  CONTROL  AND  RINGS 

The  security  kernel  software  is  responsible  for  setting  the  values  of 
the  various  fields  in  the  Descriptor  Base  Boot  and  the  descriptors. 
The  logic  of  the  SFM  is  responsible  for  checking  the  validity  of  each 
reference  using  the  descriptors  to  ensure  that  the  model's  rules  of 
security  are  followed.  The  logic  of  the  SPM  is  also  responsible  for 
computing  and  maintaining  the  current  value  of  effective  ring  number 
for  the  process. 

Access  permission  for  each  reference  is  derived  from  a combination  of 
the  R,E,W  access  bits  and  ring  brackets  shown  in  the  descriptor  with 
the  effective  rinq  number,  with  the  most  restrictive  interpretation 
controlling. 


PERFOHMANCE  CON SIDEHATICNS 

A naive  implementation  of  the  descriptor  mechanism  would  add  an 
intolerable  amount  of  system  overhead  to  each  resource  access.  Many 
references  to  main  memory  would  have  to  be  made  to  acquire  the 
descriptor  information  before  the  requested  memory  reference  could  be 
made.  Fortunately,  prior  art  has  evolved  a solution  to  this  problem., 
applicable  independently  of  the  protected  processor's  addressing 
structure.  This  is  the  use  of  an  associative  memory  cache  to  hold 
recently  used  descriptor  information,  Cn  each  reference,  the  contents 
of  the  cache  are  interrogated  first.  If  the  desired  information  is  in 
the  cache,  it  is  usea  directly.  If  the  information  is  not  in  the 
cache,  it  is  automatically  acquired  from  mem.ory  without  software 
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intervention,  added  to  the  cache,  and  used  for  the  reference. 

Iirplementa tion  techniques  are  irany  and  well  Known. 

Use  of  an  associative  iremory  adds  complexity  to  the  SPfl  but  holds  the 
performance  impact  to  acceptable  levels.  The  loss  of  performance,  due 
to  introduction  of  the  SPM,  may  be  in  the  ranqe  of  5*  to  25%  for 
typical  communications  applications  and  common  protected  processor 
architectures.  The  descriptor  cache  must  have  an  operation  time  that 
is  fast  relative  to  the  cycle  time  of  the  main  memory  in  order  to 
realize  the  full  benefits  of  its  use.  If  the  protected  processor  has 
a sophisticated  system  organization  and  already  uses  the  finest 
currently  available  technology,  any  protected  system  must  accept  the 
higher  (25%)  range  of  relative  performance  loss.  The  cbvious  source 
cf  comparative  performance  data  wciild  be  between  the  6080  and  the 
5180.  Because  of  the  totally  different  software  and  modes  of 
applications  of  the  two  machines,  no  satisfactory  performance 
comoatison  studies  of  these  machines  have  been  published.  The 
estimates  of  25%  here  are  studied  engineering  gudgements  based  on 
?lultics  and  other  experience  with  minicomputers  with  address 
translation  hardware. 

i 

3TJPPCHT  OF  MULTIPLE  PRTCSSSOR3 

There  are  at  least  three  motivations  for  multiple  processor 
configurations.  First,  it  is  to  be  expected  that  some  sites  in  a 
communications  network  may  experience  frequent  peak  load  situations  in 
which  a single  processor  is  unable  to  keep  up  with  the  workload.  This 
is  particularly  true  if  the  site's  processor  system  is  satisfying  more 
than  one  requirement  such  as  serving  as  a node  of  a communications 
network  and  also  serving  as  a front-end  processor  to  a local  host  tied 
to  the  communications  network,  or  is  itself  performing  some  of  the 
functions  of  a local  host.  In  such  a case,  it  is  advantageous  to  have 
a physical  configuration  in  which  the  workload  may  use  memory  and 
processing  facilities  wherever  available.  Second,  sites  supporting 
large  numbers  cf  terminals,  memories,  and  peripheral  equipment  may  run 
into  electrical  and  logical  limitations  of  a single  processor  bus 
system's  connection  potential,  and  require  a means  of  permitting  a 
terminal  on  one  bus  system  to  work  with  a peripheral  device  which 
happens  to  be  connected  to  another  physical  bus  system.  Third,  and 
most  important,  communications  and  similar  real  time  applications  must 
exhibit  graceful  degradation  of  performance  in  presence  of  hardware  i 

failure;  no  single  hardware  failure  should  make  any  substantial 
ccrtion.  of  the  system's  functions  unusable.  This  is  especially 
important  with  a military  system  whose  successful  operation  is  most 
important  at  times  when  it  may  be  under  greatest  physical  stress  due 
to  explosive  shocKs,  electrical  shock,  radiation,  etc. 
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Thasc  requirements  can  be  satisfied  if  a physical  module  is  available 
which  permits  interconnection  of  similar  protected  miniprocessor 
systems,  and  another  similar  module  is  available  to  permit  connection 
of  a protected  miniprocessor  to  a host  processor.  For  high 
reliability,  the  interconnection  modules  should  permit  configuring  a 
system  with  multiple  paths  available  between  each  pair  of  modules  - 
whether  processor  or  peripheral  device  or  memory  - of  the  system.  An 
example  of  such  a configuration,  interconnecting  systems  of  the  type 
described  in  Figure  18,  The  similar  processors  are  tied  together  in  a 
"delta”  formation  by  modules  referred  to  as  Inter-System  Links  (ISL) ; 
the  host  system  is  tied  in  via  Host  System  Interconnection  Units 
(HSTU).  Communications  lines,  backup  storage,  and  other  peripherals 
providing  information  uniquely  available  from  a single  source,  or 
accepting  information  uniquely  destined  for  a single  sink,  may  be 
connected  to  their  appropriate  peripheral  control  units  {BDC  or  DC  in 
the  figure)  by  multiple  paths  available  to  the  controllers.  Eevices 
for  which  only  generic  access  to  some  device  is  required,  such  as  one 
of  several  line  printers  or  term.inals  at  the  same  physical  site,  may 
be  connected  to  a single  controller.  Well  known  reliability  theory 
will  guide  this  choice. 

The  ISL  units  will  perform  a static  mapping  of  addresses  between 
busses  so  that  each  bus  occupies  a unique  area  of  the  physical  space 
(possibly  havinq  been  expanded  for  the  multi-processor  configuration.) 
This  simple  mapping  allows  each  processor  to  refer  to  the  local  bus 
Physical  address  space  independent  of  the  configuration,  yet  allows 
each  processor  complete  access  to  all  facilities  of  the  total  system. 

Use  of  multiprocessor  systems  makes  control  of  the  contents  of  the 
individual  system  cache  memories  vital.  Provision  must  be  made  to 
allow  intersystem  signaling  of  the  alteration  or  invalidation  of 
descriptors.  The  system  must  also  provide  efficient  means  of 
discovering  that  a particular  processor's  SPH  may  have  a copy  of  a 
particular  descriptor.  The  presence  of  multiple  physical  processors 
requires  additional  consideration  of  interprocess  communication's 
requirement  for  write  only  path  description.  (This  may  be  considered 
similar  to  an  I/O  process  initiation.)  The  SPfl  functions  listed  below 
will  address  these  requirements. 


ST’PPOHT  OP  INPOl/OUTFUT 


Assignment  of  Input/Output  devices  to  a process  essentially  extends 
rhe  aidress  space  available  to  the  process  to  include  the  I/O  device. 
The  control  of  this  extension  has  proven  to  be  one  of  the  most 
difficult  areas  in  the  design  of  secure  systems.  In  part,  the  problem 
I is  due  to  the  wide  variation  in  peripheral  device  characteristics  and 

) in  part  du®  to  the  unpredictably  long  times  involved  in  peripheral 
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transfers.  If  suitably  sirrple  indirection  mechanisms  can  be  defined, 
itrpletrentei,  and  inserted  between  the  process  and' the  device,  then  the 
transfer  of  inforiration  can  be  controlled  and  mediated  in  a 
certifiably  secure  manner. 

The  initiation  cf  an  I/C  activity  is  similar  to  a special  purpose 
multiprocessor  context  switch  in  which  a totally  different  processor  - 
the  I/C  controller  and  the  memory  assiqned  to  it  - is  activated  to 
operate  in  parallel  with  the  processor  requesting  the  I/O  activity. 

The  parameters  which  must  be  mediated  are  at  least  three  - 
identification  of  ^he  I/O  device,  identification  of  the  main  m.emory  to 
be  used  in  the  transfer,  ard  definition  of  the  action  (read,  write, 
control)  - all  of  whcse  consistency  with  security  must  then  be 
checked.  It  may  also  be  necessary  to  specify  other  facilities,  such 
as  a "channel"  for  the  action.  For  some  peripheral  devices,  a 
location  on  the  device  must  be  defined,  as  well  as  which  of  several 
"pseudo-devices"  or  "virtual  devices"  is  to  be  used.  The  SPM  design 
must  not  preclude  this. 

One  means  of  providing  this  indirection  and  control  is  through  use  of 
the  descriptor  based  addressing  scheme  used  for  addressing  the  memory. 
This  allows  connection  of  the  peripheral  controller  to  the  virtual  bus 
•and  provides  that  the  controller  use  only  virtual  addresses  for  its 
transfers.  The  SFM  then  mediates  each  transfer  to  or  from  memory 
during  the  translation  from  virtual  to  absolute  address.  This 
tpchnigue  treats  an  I/O  device  as  a form  of  object  with  a virtual  name 
completely  analogous  to  a virtual  memory  address. 

Ar.  alternate  m.eans  is  to  have  the  SPM  mediate  the  initial  value  of  the 
address  to  be  used  and  the  operation  to  be  performed  for  a peripheral 
transfer  and  to  have  it  set  a limit  on  the  extent  of  the  transfer.  It 
is  then  the  responsibility  of  the  controller  - operating  on  the 
"absolute"  memory  bus  - to  remain  within  the  bounds  set.  To  insure 
that  the  controller  can  be  trusted  to  obey  the  security  rules,  the 
specification,  design,  and  implementation  of  such  controllers  will 
have  to  be  certified  in  much  the  same  way  as  the  CPU  and  SPM  hardware 
is  to  be  certified.  Such  a technique  allows  connection  of  the 
peripheral  controller  on  the  absolute  bus.  This  is  analogous  to  the 
creation  of  a special  purpose  process  with  a separate  identity, 
operating  on  ar  1/0  controller  as  its  processor.  The  pre-transfer 
mediation  is  analogous  to  the  kernel’s  creation  of  a new  process  in 
that  the  kernel  must  insure  that  the  process  is  initially  in  a secure 
3ta*-e  and  has  only  valid  permissions-  The  initiation,  control  and 
termination  of  an  I/C  transfer  is  analogous  to  the  initiation, 
scheduling  and  termination  of  a normal  process. 

Two  methods  of  handling  peripheral  device  names  and  physical  addresses 
have  evolved.  One,  the  "explicit  names"  approach,  associates  a 
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virtual  nama  with  each  physical  device  available  to  the  systeir  (and 
with  each  virtual  device  as  well).  The  other,  the  "devices  as  ireir.ory" 
approach,  associates  a set  of  virtual  memory  locations  with  each 
device.  Either  of  these  techniques  may  be  used  with  the  descriptor 
based  addrossinq  schema  used  by  the  SPH;  aither  is  consistent  with 
security  actions  of  any  existinq  peripheral  device. 

Termination  of  an  I/C  transfer  results  in  an  interrupt  to  the  system. 
This  initiates  an  automatic  context  switch  according  tc  properties  of 
the  protected  processor,  as  augmented  by  the  SPI3  to  ensure  security. 
These  actions  will  be  the  same  as  considered  above  in  Section  IV's 
overview  of  sacure  context  switching  requireirents. 


SnPPCRT  FCP.  THE  INITIAL  SECUFE  STATE 

The  fundamental  security  models  deal  with  security  preservation  - the 
croblem  of  heepinq  the  system  in  a secure  state  assuming  that  it  was 
initially  in  a secure  state.  One  of  the  problems  tor  the  system 
designer  is  to  establish  the  initial  secure  state  of  the  system. 
Systems  are  traditionally  started  up  by  use  of  a bootstrap  routine  or 
initial  proqrair  load  procedure  of  some  sort.  It  is  essential  to  tne 
security  of  the  Secure  Communications  Processor  that  a means  be 
provided  to  establish  this  initial  state  with  great  confidence. 

Both  the  SPh  and  the  security  kernel  (in  ring  0)  irust  be  initialized 
to  a secure  state.  It  is  suggested  that  as  much  of  this  initial  state 
as  practical  be  established  by  use  of  a Read-Only  Seirory  (ROM)  unit 
placed  within  the  SPH.  Physical  protection  of  the  initializing 
function  may  be  provided  by  use  of  a key  lock,  combination  lock,  or 
some  other  such  device  to  prevent  unauthorized  activation  of  the 
secure  initialization  procedure.  The  initialization  will  also  use  the 
secure  context  switch  and  other  facilities  to  create  a new  process,  as 
mentioned  in  iv  above. 
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SECTION  V 


aEQOIREHENTS  POP.  THE  PPOTECTED  SYSTEM 


INTPCDMCTION 

This  section  will  consider  design  requirements  for  the  components  of 
the  system  used  to  build  a Secure  Communications  Processor.  First  the 
control  requirements  of  the  SPM  will  be  discussed  and  listed.  Then 
♦■he  functional  requirements  for  the  CPU,  Memory,  and  Peripheral 
Con^-rollors  of  the  protected  system  will  be  covered.  Finally,  some  of 
the  functional  requirements  of  the  associative  or  cache  memory  for  the 
SFM  will  be  considered. 


SPM  IMST5UCTICNS 

There  will  be  a set  of  instructions  cr  functions  used  by  the  security 
kernel  in  controlling  the  SPM,  These  instructions  must  be  executed, 
in  part,  by  the  CPU  that  is  suoportinq  the  security  kernel  process, 
■’’hese  instructions  must  be  restricted  to  operate  only  when  the  CPU  is 
in  Ring  0 so  that  only  the  security  kernel  may  issue  them-  Host 
activities  of  the  security  kernel  will  not  require  special 
instructions  or  supoorting  actions  of  the  SPM.  For  example,  the 
creation  or  modification  of  a descriptor  may  be  done  by  the  security 
kernel  without  special  aid  from  the  SPM,  The  SPM  will  be  involved  in 
such  an  action,  but  only  to  the  extent  that  it  is  involved  in  any 
reference  to  memory  by  any  process. 


The  minimum  set 
designer  of  the 
desirable  or  he 
different  ways, 
with. 


of  SPM  control  instructions  is  outlined  below.  The 
system  may  find  that  some  additional  instructions  are 
may  choose  to  implemient  these  instructions  in 
depending  on  the  nature  of  the  system  he  is  working 


Clear  Associative  Memories 


This  instruction  will  effectively  clear  or  invalida 
the  soecified  associative  (cache)  memory  of  the  SPM 
descriptor  information  as  specified.  The  logic  of 
loading  the  contents  of  the  cache  memory  as  part  of 
mediation  activity.  This  command  gives  the  securit 
over  the  removal  of  such  information  from  the  memor 


te  the  contents  of 
, removing 
the  SPM  will  handle 
its  address 
y kernel  control 

y* 
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At  least  two  variables  are  nee>ied  for  this  command  to  allow 
specification  of  the  extent  of  the  command.  The  variables  are: 

a.  SP1  Address;  The  command  must  be  able  ro  operate  on  all  SPM's 
in  the  system  or  iust  on  the  SPH  associate!  with  the 
processor  that  issued  the  command. 

b.  Descriptor  Address;  Clear  all  descriptors  from  the  addressed 
SPPi  or  remove  these  descriptors  specified. 

The  ability  to  remove  scire  specified  subset  of  all  descriptors  must  be 
implemented  in  a fail  safe  manner,  leavinq  only  those  descriptors 

which  cannot  possibly  be  "SIMILAR  to"  any  descriptor  (s)  specified,  ; 

where  "SIMILAF."  is  fail  safe,  and  otherwise  defined  for  convenience  of  \ 

implementation.  This  facility  will  be  a necessary  time  saver  if  SPM  i 

descriptor  storage  is  large  and  the  system  has  several 

multiprocessors;  it  may  be  unnecessary  if  analysis  proves  that  a small  ; 

descriptor  storage  in  the  SPM  is  adequate.  | 

2.  Furnish  Unique  Identifier 

This  instruction  or  function  will  cause  the  SPM  to  generate  a unique 
value  usable  to  identify  a process  or  segment.  This  instruction  could 
be  performed  instead  by  software  in  the  security  kernel,  if  there  can 
be  provision  for  reliable  storage  of  the  generating  parameters.  The  !) 

reauirement  is  for  an  unique  process_id  with  no  possibility  of 
duplication  over  a long  time  span  exceeding  expected  s/stom  life,  even 
if  the  system  is  stopped  and  powered  down  repeatedly. 

The  Multics  system  contains  a hardware  clock,  accessible  by  the  CPU, 
which  counts  micrcseconds  since  Midnight  (GMT)  January  1,  1900.  The  \ 

clock  will  provide  unambiguous  counts  until  late  in  the  year  2042  AD,  d 

and  provides  urigue  identifier  bit  strings  for  use  in  processor  and 
message  TD's.  The  resolution  of  1 microsecond  prevents  two  accesses  1-i 

of  the  same  court.  The  period  of  142+  years  assures  non-repetiticn. 

A similar  counter  would  suffice  for  the  Secure  Communications 
Processor. 

3..  Save  Context;  Restore  Context 

These  instructions  and  functions  will  cause  the  SPM  to  store  (and 
restore)  the  registers  and  control  information  needed  to  specify  the 
status  of  a process.  Among  the  items  ^o  be  stored  are  the  current 
effective  ring  number.  Descriptor  Base  Foot,  descriptor  fault 
identification  information,  instruction  ani  address  evaluation 
progress  ("depth")  information  at  last  fault,  and  various  other 
*)  internal  security  related  items.  The  information  must  be  stored  in 
memory,  accessible  only  to  the  security  kernel  running  in  Ring  0, 


65 


( 


These  functions  will  auto 
context  instructions  of  t 
from  a security  point  of 
not  be  able  to  bypass  or 
vital  to  the  performance 
the  CPU  and  SPK  operate  a 
that  system  overhead  will 


rratically  auqment  the  save  and  restore 
he  CPU,  handlinq  all  iteirs  that  are  sensitive 
view.  It  is  essential  that  the  user  software 
circumvent  this  procedure  in  any  way.  It  is 
of  the  system  that  the  combined  actions  of 
s quickly  as  possible  for  a context  switch  so 
be  minimized  in  these  frequently  used  areas. 


It  is  important  to  note  that  fault  response  processing  is  likely  the 
most  frequent  route  to  the  kernel.  The  system  should  make  it  easy  to 
save  and  restore  the  minimum  amount  of  information  in  these  cases- 
Implementation  which  does  not  require  clearing  all  registers  of  the 
protected  processor  will  not  only  save  time,  but  also  avoid  forced 
removal  of  information  necessary  to  the  kernel's  functions.  The 
inclusion  of  a multiple  save  and  clear  registers  and  the  complementary 
load  multiple  registers  instructions  will  allow  the  kernel  to  decide 
the  disposition  of  a trap  before  completely  changing  context,  yet 
efficiently  exchange  contexts  if  a domain  change  is  actually  reguired, 

u.  Test 

A comprehensive  set  of  SPM  testing  instructions  is  needed  so  that  the 
security  kernel  may  dynamically  verify  that  the  system  is  meeting  the 
security  rules.  The  test  instruction  set  should  allow  testing  of 
functions  of  the  SEM,  interconnections  of  the  system,  and.  major 
components  of  the  system.  These  instructions  will  also  be  of  use  in 
diagnosis  of  hardware  problems  and  in  monitoring  the  system  for 
hardware  failure.  About  5%  to  10%  extra  SPM  hardware  and  firmware 
must  be  allowed  to  implement  adequate  fault  detection  and  diagnostic 
functions. 

5,  Ring  Crossing 

The  operation  of  changing  a process  from  one  ring  of  executio;i  to 
another  and  returning,  without  compromising  security,  may  be  borrowed 
from  the  Multics  system.,  A Call  instruction 

Call  (gate  entry,  procedure_name , parameters) 

specifies  in  one  of  its  arguments  that  a "gate"  descriptor  array  is 
needed.  The  current  operating  ring  r must  satisfy  the  inegualities 

R2  < r < R3  (gate  parameter  descriptor) 

or  a call  fault  will  be  signaled.  The  parameters  P2  and  R3  are  the 
read  and  call  bracket  values,  respectively,  for  the  current  process. 
The  new  effective  ring  of  execution  r'  is  the  most  secure  of  the 
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present  rinq  of  execution,  r,  and  the  Write  bracket  of  the  descriptor 
used  to  pass  the  parameters  to  the  new  execution  rinq,  namely, 

new  rinq  r*  = n-in(r,a2  of  procedure_name  descriptor) 


In  cross  rinq  calls,  arqutrent  validation  is  an  important 
consideration.  It  is  important  that  the  called  routine  (at  a higher 
privilege)  access  no  data  passed  as  parameters  that  the  calling 
prooram  cannot  access.  Likewise  upon  return,  the  calling  program  must 
not  be  passed  data  to  which  it  has  no  access  rights.  Call  by  value 
arqumants  present  no  problem  since  the  S?M  mediates  their  access  prior 
to  the  call  in  the  action  of  putting  the  arguments  in  the  calling 
sequence.  The  problem  comes  from  indirect  or  call  by  reference 
arguments  which  are  not  actually  mediated  until  used  and  will  be  used 
at  a higher  privilege  rinq. 

Several  possible  solutions  exist.  The  indirection  constant  may  carry 
the  caller's  ring  and  access  rights  established  prior  to  the  call. 

Then  the  SPM  can  temporarily  use  these  to  check  the  actual  fetch  cf 
the  data  by  the  called  program.  Inis  requires  a special  form  of 
indirection  and  a special  SFM  mode  to  mediate  the  pointer  using  the 
proper  rincr  and  access  rights.  Theoretically,  the  called  program, 
given  access  to  the  caller’s  rinq  and  access  rights,  could  be  depended 
upon  to  check  all  accesses  via  software.  This  poses  a significant 
burden  by  requiring  the  verification  of  all  called  routines  and  would 
add  an  intolerable  performance  penalty. 

If  multi-level  indirection  is  available  in  the  hardware,  the  caller 
can  pass  a pointer  to  an  indirection  constant  in  the  callers  ring.  At 
actual  mediation  time,  the  SPM  will  mediate  each  indirection  pointer 
using  the  Lescriptor  Base  Foot  of  the  rinq  in  which  the  pointer 
exists.  This  requires  extra  mediation  by  the  SPM  for  each  access  and 
requires  multi-level  indirection  in  the  hardware  with  the  attendant 
problems  associated  with  unbounded  levels  of  indirection. 

A third  technique  requires  a special  instrtiction  which  causes  the  SPM 
to  validate,  the  access  rights  without  the  CPU’s  perform, inq  the  memory 
access.  This  instruction  allows  the  kernel  to  have  the  SPM  perform 
the  mediation.  If  the  mediation  is  successful,  the  kernel  can  then 
copy  the  reference  into  the  called  domaiii  for  further  use  without 
multi-level  indirection. 

The  choice  of  actual  method  of  argument  validation  is  dependent  on  the 
design  of  the  CPU  and  SPM.  The  designers  may  use  one  of  these  methods 
or  another  equivalent  method,  but  one  must  be  provided. 
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Automatic  Data  Storaqe  - Faults  and  Interrupts  ("Traps") 


Table  I summarizes  information  which  .the  protected  processor  and  the 
Security  Frotectior  Module  together  must  ensure  is  saved  whenever  a 
Fault,  Interrupt  or  other  "trap"  event  occurs.  The  critical  items 
from,  the  security  point  of  view  are  marked  with  a double  asterisk 
{**) . '^able  II  summarizes  an  acceptable  content  of  fault  data  words 
in  the  SFM. 

7.  Passage  of  effective  rings  and  process_id. 

When  I/O  operations  are  begun,  or  any  other  operation  is  performed 
activating  another  physical  process  to  operate  independently,  the  SFM 
must  be  sure  that  the  effective  access  ring  number  and  ptocfess_id  are 
passed  in  user  unalterable  form.  From  the  analytic  point  of  view, 
this  is  merely  a special  case  of  secure  process  creation,  but  it 
requires  care  in  practice. 


TABLE  I 

FAULT  r.  TFAP  (RARE  EVENT)  IKFOFMATION 

This  summarizes  the  information  stored  ty  the  protected  processor  and 
the  SPM  during  a trap  due  to  fault,  interrupt,  or  other  rare  events. 

The  information  to  be  stored  in  a trap  frame  includes  at  least  the 
following: 

For  a processor  related  rare  event: 


♦the  instruction  (INST) 

♦the  effective  address  cf  this  (or  the  next)  instruction 
(Program  Counter) 

♦the  effective  address  (EA)  being  processed  at  the  time 

♦identification  of  this  address  within  the  instruction 
(1st, 2nd,  etc) 

♦the  indirection  level,  including  each  descriptor  or 
indirect  address  as  a new  indirection  level; 

♦cause:  data,  processing,  address,  or  description; 

♦♦execution  ring,  u£€r_id 

♦other  user  visible  trap  defining  information; 

♦ ♦security  related  trap  defining  intoriration 


♦whether  an  automatic  processor  context  save  was  performed 
by  this  trap; 

♦if  context  saved,  pointer  to  its  location; 

♦where  the  next  trap  frame  may  be  stored,  if  necessary 
♦where  the  previous  trap  frame  was  stored,  if  reguired 
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For  a peripheral  related  rare  event  (interrupt) : 

The  instruction,  operands,  and  other  particulars  relating  to  the 
instruction's  effective  address  being  executed  at  the  time  of  the 
interrupt  are  irrelevant.  Interruption  may  be  delayed  until  the 
termination  of  the  next  instruction  at  which  an  interrupt  is 
acceptable.  Instead  of  these,  there  must  be 

* identification  of  the  peripheral  device  responsible  for  the 
interrupt ; 

* identification  of  the  reason  for  the  interrupt; 

* status  of  the  peripheral  device  at  tiire  of  the  interrupt; 

* status  of  its  peripheral  transfer  process  at  time  of  the 
interrupt. 


TABLE  II 


SECURITY  RELATED  FAULT  INFORMATION 
fault  data  word  content 

The  following  irfortration  is  stored  in  the  two  words  reserved  for  SPM 
fault  data  collection: 


Word  1 : 


Word  2 : 


3it 

0: 

Hardware  fault 

Bit 

1: 

Hardware  fault 

Bit 

2: 

Illegal  user_id 

Bit 

3: 

Illegal  user_id 

Bit 

a: 

Ring  > 0,  Ring  0 address 

Bit 

5: 

Ring  > 0,  Ring  0 privileged 

instruction 

Bit 

6: 

Reserved  for  Future  Use 

Bit 

7: 

RFU 

Bit 

8: 

Directed  fault 

Bit 

9: 

Directed  fault 

Bit 

10: 

Limit  exceeded  (no  space) 

Bit 

11: 

Limit  exceeded  (no  space) 

Bit 

12: 

Read  fault 

Bit 

13: 

Execute  fault 

Bit 

14: 

Write  fault 

Bit 

15: 

Call  fault 

«it 

16: 

Permission  missing 

Bit 

17: 

Permission  missing 

Bit 

18: 

Bracket  improper 

Bit 

19: 

Bracket  improper 

Bit 

20: 

CALL  gate  reguired 

Bit 

21: 

RFU 

Bit 

22: 

RFU 

Bit 

23: 

Indirection  depth  exceeded 

(if  not  in  26 

Bit 

24: 

Multi  process  usage  limit  exceeded 

Bit 

25: 

Multiprocess  semaphore  access  fault 

Bits 

26- 

31:  reserved  for  depth  indicators 

(Details  of  the  format  and  content  of  this  list  are  illustrative.  The 
actual  usage  and  definitions  will  depend  on  actual  hardware  and 
software  irrpleirentation) . 
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PHCCE3S01  R2AIIZATICM  REQDIFEMENTS 


Coirpatibilitv  with  communications  reauiras  a processor  capable  of 
efficiently  hardlinq  bytes.  Maximum  efficiency  combined  with  maximum 
availability  of  commercially  modifiable  peripheral  and  memory  modules 
suqqests  that  two  bytes  be  handled  in  parallel.  Maintenance  of 
maximum  potential  compatibility  with  existinq  equipments  suggests  tnat 
3 bit  bytes  are  a must,  that  6 bit  bytes  may  be  a frequently  required 
subset,  and  that  9 bit  bytes  should  not  be  precluded. 

Parity  checking  is  a irust  if  early  recognition  of  hardware  failure  is 
to  be  achieved,  and  other  types  of  hardware  checking  should  be  used 
wherever  available  and  mandatorily  provided  for  obviously  security 
critical  circuitry.  The  need  for  creating  operation  codes  reserved 
for  the  reference  monitor's  use  alone  implies  a processor  with  a 
vocabulary  of  unused  "generic"  or  "unassigned"  or  "user  assignable" 
operation  formats. 

\ processor  connected  with  its  peripheral  and  memory  modules  by  a 
standard  type  of  interface  for  a large  sat  of  existing  equipments  is  a 
must.  The  bus  design,  moreover,  must  permit  asynchronous  operation, 
to  tolerate  the  delays  required  in  implementation  of  protection 
checking. 

\ fast  context  switching  facility  is  required,  if  performance 
bottlenecks  found  with  a previous  study  (Reference  7)  are  to  be 
avoided.  Means  of  identifying,  in  an  unforqeable  manner,  which 
module,  and  which  subunit  within  a module,  is  requesting  access  to  the 
bus,  must  be  present  or  easily  providable.  All  private  storage, 
including  any  cache  locations,  irust  be  capable  of  complete  erasure  by 
external  action,  to  prevent  leakage  of  information  between  processes 
in  context  switching.  A real  time  clock  is  needed  to  ensure  periodic 
entry  to  the  reference  monitor;  it  also  assists  the  generation  of 
unique  identifications  not  repeatable  in  the  system's  life. 


These  features  may  be  summari2ed  as: 

• byte  compatibility 

♦ two  byte  data  path  width 

♦ 9 bit  byte  potential 

* parity  checking 
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* other  checkinq 

* standard  bus  for  interrrodule  cotmnunication 

* asynchronous  operation 

* fast  context  switchinq 

* potential  for  unforqeable  module  and  submodule  identification 

* reserved  operation  codes’ 

* available  peripherals  for  com.irunications,  disks,  tapes, 
printers,  and  a larqe  variety  of  other  standard  equipments 
compatible  with  present  qovernment  usaqes. 

* real  time  clock  > 

* interruption  turn  off 

* uninterruptible  increment/decrement  or  test  and  set  (for 
semaphores) 

Required  Certified  Operations 

he  kernel  will  require  a subset  of  instructions  verified  to  do 
xactlv  what  they  are  specified  to  do,  and  a means  of  using  these  to 
change  each  component  of  the  maps.  This  subset  should  be  kept  as 
small  as  possible  to  permit  verification.  The  suggested  set  is 
summarized; 

Load  to  working  reqis-ter(s)  from,  memory. 

Store  to  memory  from  working  register  (s) . 

(Notation;  Txl  f contents  of,  "op”  = operation, 

EA  = "Effective  Address”, 

WR  = "Horkinq  Register",  IND  branch  indicators.) 

Arithmetic  to  working  registers: 

fWRl  op  teat  — > WR,  with  op  = ♦,  -,  *,  /,  Modulo  (small 
operands)  ; 

fE\l  ♦ 1 fEAl  (increment  or  decrement  semaphore). 

Logic: 

rwRl  op  fEAl  — > fWRl  with  op  = 6,  |,  (fc; 

rwsi  ::  fEAl  — > IND  (comparison  with  storage). 

Branch  on  IND  conditions;  all  combinations  of  +,  -,  = 
or  equivalent  indicators. 

I/O;  Read,  write,  control  op  cr  status  inf orm.ation. 
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It  would  also  be  convenient,  altbouqh  not  absolutely  necessary,  to 
take  advantage  of  frequently  used  coirposites  of  these  which  are  often 
available  on  modern  minicomputers.  Certified  instructions  to  swap  HP 
and  memory,  and  instructions  to  provide  loading  and  storing  with 
protected  masking  of  the  memory  contents  (to  manipulate  arbitrary  bit 
fields  in  descriptors)  would  greatly  shorten  the  time  spent  in  the 
Reference  Monitor  software.  A means  of  providing  the  equivalent  of  a 
test  and  set  is  also  required;  this  may  be  accom.plished  by  the 
uninterruptible  execution  of  a com.bination  of  other  verified 
instructions  of  the  subset,  or  by  a separate  verified  instruction. 


A certified  facility  is  also  needed  to  establish  efficient 
addressability  to  selected  descriptors  of  an  address,  rather  than  the 
data  ordinarily  described.  There  must  also  be  a quick  means  of 
establishing  efficient  and  certified  access  to  fast  access  working 
space  for  descriptor  manipulation.  These  may  well  be  found  inherent 
in  practical  facilities  of  the  protected  processor  and  its  interfaces 
to  the  SFM. 


RSALI2ATI0;i  RFQniFEMiNTS  - MAIN  MEMCRY 


The  main  memory  modules  for  use 
Processor  should  be  basically  a 
following  properties  to  provide 


with  the  Secure  Communications 
standard  comm.ercial  design,  with 
maximum  utility  and  performance. 


the 


Error  Cetection  and  Correction 


The  reliability  requiremen.ts  are  high,  especially  since  unreliability 
results  in  a substantially  greater  risk  of  breach  of  security. 
Therefore  every  reasonable  reliability  increasing  practice  should  be 
used.  The  provision  of  Error  Detection  and  Correction  (EDAC)  in  the 
memory  modules  is  one  of  the  most  useful  techniques  for  extending  the 
Mean  Time  Between  Failure.  An  increase  of  the  order  of  a factor  of 
six  between  necessary  repairs,  for  the  same  probability  of  failure  per 
urit  time,  is  expected  on  one  typical  design.  Moreover,  this 
technique  gives  warning  of  potential  failure  long  before  the  failure 
is  actually  experienced  ir  most  cases.  Hence  error  detection  and 
correction  is  strongly  recommended  for  a Secure  Communications 
Prccessor  system. 


Read- Alter-Write  Memory  Cperation 

There  are  a number  of  circumstances  in  descriptor  manipulation  and 
secure  certifiable  queueing  in  waich  it  is  mandatory  that  information 
be  removed  from  the  main  memory,  and  returned,  in  altered  form, 
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without  the  possibility  of  another  user  - such  as  a peripheral 
crocessor  or  another  processor  of  a irultiprocessor  system  - being  able 
to  access  the  same  resource  in  the  meantime.  Manipulation  of  the  U, 

C,  F,  WCT,  I fields  of  descriptors  are  cases  in  point.  This 
uninterruptible  manipulation  is  also  important  in  increnrentinq  or 
decrerrentinq  semaphores  (DiiXstra's  "P"  and  '*V"  operations)  for 
properly  secure  queueirg  and  dispatching  (processor  assignment) . ?his 
facility  must  be  built  into  the  memory  module  for  maximum  efficiency 
and  usability. 

Module  Identification 

Security  requires  that  each  module  connected  to  the  absolute,  or 
certified,  bus,  be  able  to  furnish  an  unforqeable  identification  or 
itself  at  each  access.  The  memory  module  is  particularly  important  in 
this  regard,  especially  when  it  is  responding  to  requests  for  service 
made  some  time  previously. 

Papjd  Standardization  of  Me.morv  Contents 


A secure  system  operate 
any  time.  In  order  to 
another,  it  will  clear 
such  as  zero,  immediate 
waiting  for  new  guantit 
released  memory.  There 
memory  can  be  zeroed  or 
standard  value  with  qre 
carefully  considered  si 


s on  the  assumption  that  it  may  "go  down”  at 
prevent  leaXaqe  of  information  from  one  user  to 
each  section  of  memory  to  a standard  value, 

Iv  upon  each  release  of  facilities,  without 
ies  to  be  assigned  and  written  ever  the 
are  memory  modules  in  wh-ich  large  oarts  of  the 
otherwise  returned  to  an  uninformative 
at  speed.  Such  memory  modules  should  be 
nee  all  unassiqned  memory  should  be  cleared. 


REALISATION  HECOIBEMENTS:  PEBIPHEEAL  CONTP.OLLEHS  AND  DEVICES 

The  ma1or  requirement  is  to  provide  for  operation  of  all  the 
peripheral  devices  and  controllers  needed.  These  include  all  common 
types  - especially  communications  ccntrollers. 

The  controllers  themselves  must  be  certifiable  to  perform  exactly  as 
requested  in  at  least  the  following  ways: 

♦ responding  to  the  correct  controller  identification,  and  no 
other ; 

♦ selecting  the  proper  device,  and  no  other; 

♦ selecting  the  proper  location  on  the  device  (where 
appropriate)  and  no  other; 


* performinq  the  proper  action  with  the  aerrice,  and  no  other. 

In  particular,  no  confusion  trust  be  made  possible  between  read 
and  write,  and  between  a request  for  data  action  and  a request 
for  control  action. 

If  the  controller  is  to  operate  on  a bus  shared  with  the  iremory 
modules  - an  "absolute”  or  "certified"  bus  - the  followinq  certified 
correct  properties  are  also  necessary; 

♦ correct  acceptance  of  absolute  addresses  furnished  to  be  used 
in  peripheral  data  transfer  or  other  control  usage; 

♦ correct  manipulation  of  absolute  addresses  furnished  for  use 
in  peripheral  data  transfer,  or  control  usage; 

♦ correct  termination  of  a data  transfer  at  or  before  the  point 
where  assigned  memory  space  has  been  exceeded; 

* correct  presentation  of  a request  for  "write"  or  "read"  to  the 
memory  bus; 

* requesting  action  of  the  bus  only  when  previously  instructed 
to  begin  an  action  requiring  it. 

Each  controller  must,  in  any  case,  either  furnish,  or  be  modified  to 
furnish,  an  urforqeable  unique  identification  of  itself  and  the  device 
in  use  to  accompany  each  request  for  service  on  the  bus.  Controllers 
of  devices  which  typically  must  be  shared  among  users,  such  as  disks 
or  drums,  should  be  designed  or  connectable  to  permit  definition  of 
pseudo-devices  comprising  arbitrarily  selected  combinations  of 
locations  on  the  device. 


CACHil  MEMORY  PEfiLIZATICN  CONSIDEBftTIONS 

There  are  three  principal  design  decisions  in  the  implementation  of 
the  cache.  The  first  decision  is  the  method  of  operation.  The  cache 
will  be  made  of  a fast  access  memory,  of  course,  but  there  are  two 
principal  design  choices.  One  is  to  use  a true  associative  memory,  in 
which  any  quantity  may  be  stored  in  any  location  and  recovered.  Such 
memories  provide  a higher  "hit"  ratio  - that  is,  probability  of 
finding  a quantity  desired  already  in  the  cache  - per  word  stored, 
because  there  are  fewer  restrictions  in  selecting  a location  to  be 
rewritten  when  a new  quantity  must  be  referenced.  On  the  other  hand, 
they  tend  ♦■o  be  slower  and  somewhat  more  expensive.  The  other 
alternative  is  to  design  a "pseudo-associative"  cache  cf  standard  fast 
access  memory,  using  locations  within  the  cache  to  perform  part  of  the 
association  by  choosing  a subset  of  the  bits  of  the  associated 
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quantity  to  address  them,  and  usinq  comparison  loqic  to  perform  the 
remainder  of  the  association. 


Cache  Size  | 

i I 

Another  choice  in  implementation  is  the  size  of  the  cache.  Previous 
work  has  suqqested  the  maximum  possibly  useful  descriptor  storage  to 
be  of  the  order  of  512  descriptor  positions,  and  that  a number  of  the 
order  of  6*i  to  128  may  be  adequate  in  practice  (Reference  20)  . If 
Dure  associatiye  realization  is  used,  the  qreater  flexibility  of  the 
replacement  choices  with  a genuine  associator  nay  make  as  few  as 
sixteen  per  user  a highly  effective  choice,  (10)  Since  it  is  expected 
that  there  mav  be  frequent  switches  to  the  Reference  Monitor  software, 
it  is  very  advantageous  to  design  the  cache  to  be  large  enough,  and 
with  an  associating  algorithm  so  chosen,  that  the  reference  monitor 
and  the  system  code’s  most  recently  used  descriptors  are  very  likely 
not  to  use  the  same  space  as  the  user's  descriptors.  This  can  be 
done,  for  example,  by  providing  at  least  sixteen  descriptor  storages 
oer  rinq,  and,  if  a pseudo-associa tor  is  used,  connecting  the  bit(s)  n 

of  ^he  associative  address  which  d iscriminate  (s)  between  the  reference 
monitor  system  code  and  user  code,  to  be  part  of  tne  bits  wired  to 
select  cache  locations  directly,  rather  than  part  of  the  bits  wired  to 
the  comoarison  loqic.  For  an  explanation  of  operation  of  one  specific 
pseudo-associative  cache  see  Reterence  33,  pages  7-1  tc  7-4. 


Descriptor  Bits  Not  Mandatorilv  Represented  in  the  Cache 


A third  implementation  consideration  is  which  fields  of  the  descriptor 
need  be  explicitly  represented  in  the  cache.  Those  which  represent 
information  of  use  only  ir  main  memory  need  not  be  so  represented. 

The  following  bits  of  Figure  13  must  be  represented  explicitly: 

?1,.R2,R3 
R,E,W,M 
Base  location 
Sase  location  extension 
D (type) 

Limit 


I (■’0)  Multics  experiments  (Reference  20)  indicated  a small  but 

measurable  difference  between  a 1 6-associator  and  an  8-associator 
^ descriptor  storage. 
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Other  hits  need  rot  necessarily  be  represented  within  the  fast  access 
descriptor  cache  storaqe,  but  may  be  represented  if  other  design 
considerations  dictate.  The  bits  describing  whether  the  descriptor 
has  been  used  (!I)  , is  encacheable  (C)  , has  been  locked  into  memory 
(L) , or  how  many  users  have  "wired”  it  (WCT)  , are  all  needed  only  in 
algorithms  involving  main  memory  communication  between  processes  and 
processors.  These  kits  and  the  unused  bits  may  be  eliminated  from  the 
cache  storaqe  if  economical.  However,  their  main  memory 
representations  are  consulted  and  changed  by  SFM  firmware  algorithms. 

It  is  noteworthy  that  explicit  representation  of  the  first  two  words 
of  the  descriptor  in  cache  prepares  the  way  for  a very  efficient  data 
cache  design  as  well.  Explicit  representation  of  the  first  three 
words  of  the  recommended  descriptor  format  creates  a storaqe  format 
also  useful  for  I/C  descriptor  storaqe  if  the  limit  and  base  fields 
are  extended  to  describe  single  words  during  the  I/O  data  transfers. 

In  addition  to  the  encachement  of  descriptors  as  considered  above, 
other  information  may  be  included  in  a cache.  A data  cache  increases 
the  effective  cycle  speed  of  main  memory.  A data  cache  in  the  SPK, 
added  to  a processor  which  possesses  none,  may  conveniently  recover 
much  performance  cost  of  the  protection. 

The  D3P  (Descriptor  Base  Foot)  might  also  be  encachcd.  A small 
conventionally  indexed  storaqe  system  of  perhaps  16  words  would 
probably  suffice.  Since  the  DER  storaqe  is  expected  to  be  consulted 
infrequently,  it  is  not  important  that  all  of  its  contents  be 
available  in  parallel.  It  may  also  be  stored  in  a general  descriptor 
cache  location. 

Details  of  the  cache  implementation  must  be  the  responsibility  of  the 
implementor;  these  guidelines  may  help.  The  best  economical 
approximation  to  an  optimum  replacement  algorithm  is  also  a function 
of  this  cache  implementation. 
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SECTION  VI 

IMPLEMEKTATION  CCNSIDEEATIONS 


INTRCCDCTICN 


This  section  is  concerned  with  a number  of 
First,  there  are  soire  estimates  of  the  phy 
SPM,  usinq  different  system  structures.  T 
recommendations  and  tradeoff  discussion, 
reliability  is  considered.  The  estimate  o 
security  breach  due  to  hardware  failure  is 
presented  of  the  analysis  to  estimate  secu 
a practical  system.  The  section  ends  with 
performance  of  a protected  system  used  as 
Processor. 


implementation  areas, 
sical  size  of  a practical 
his  leads  to  some  design 
Next,  the  subject  of  system 
f the  risk  per  unit  time  of 
discussed.  An  outline  is 
rity  breach  per  unit  time  on 
an  attempt  to  predict  the 
a Secure  Communications 


The  princioal  implementation  conclusions  can  be  summarized  as  follows; 

♦ The  SPM  can  be  built  for  the  approximate  cost  of  a pare 
miniprocessor  without  memory  or  peripheral  equipment  or  console. 
This  is  a small  fraction  - approximately  lOji  or  less  - of  a 
typical  minicomputer  system's  hardware  cost. 

♦ Implementation  usinq  cache  memory  techniques  and  a variaole 
size  segment  descriptor  rather  than  a paqe  table  is  preferable 
for  system  growth  and  flexibility  of  application.  Moreover,  this 
technique  costs  no  more  than  others,  either  in  performance  or  in 
hardware. 

♦ Installing  an  SPM  in  most  minicomputer  systems  will  iirply  a 
performance  loss  of  approximately  to  254.  This  is  a small 
fraction  of  the  ratio  of  system  performance  ordinarily  expected 
between  a particular  miniprocessor  model  and  the  next  most  costly 
compatible  model  of  tne  same  manufacturer's  line. 
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* While  pre-existinq  software  may  be  used  on  the  protected 
system  without  chanqe  if  the  operatinq  system  and  kernel  software 
are  properly  desiqned,  quidelines  can  be  furnished  to  increase 
the  efficiency  of  common  system  or  application  proqrams  in  a 
secured  environment.  This  is  particularly  true  in  a 
multiprocessor  environment,  which  is  preferred  for  hiqh 
reliability, 

* Imnlemert ation  usinq  standard  minicomputer  boards,  subsystems, 
components,  and  bus  orqanization  is  practical. 

* A useful  model  of  the  probability  per  unit  time  that  hardware 
failure  causes  a security  breach  can  be  developed  in  terms  of 
well  known  characteristics  of  any  miniprocessor  * s subsystems  - 
The  Mean  Time  Between  Failure  (MTBF) , and  the  loqical 
orqanization. 

* An  upper  limit  on  the  rate  of  security  breach  due  to  hardware 
failure  can  be  specified.  It  appears  practical  to  obtain  the 
order  of  a million  system  hours  of  operation  between  expected 
hardware  caused  security  breach. 


ESTIMATES  CF  SIZE  ANC  COST 


Estimates  of  relative  size  and  cost  of  a special  purpose  processor  and 
oeripherals  vs.  a special  purpose  security  unit  conclude  that  the 
descriptor  processinq  facility  will  be  of  the  order  of  maqnitude  of  a 
present  day  miniprocessor,  requirinq  at  least  500  inteqrated  circuit 
oackaqes  ("IC's")  usinq  current  technoloqy,  to  manufacture  both  the 
processor  and  its  necessary  security  supportinq  equipment.  This 
aqrees  with  the  .'lultics  processor  desiqn  experience,  where  the 
lescriptor  manipulatinq  equipment  required  the  equivalent  of  another 
arithmetic  unit  - about  25%  of  the  processor  subsystem. 

with  the  packaqinq  techniques  available  today,  about  three 
miricomputer  boards  will  be  necessary  to  hold  all  of  the  integrated 
rircuitrv  needed  to  implement  a special  purpose  security  unit  on  a 
;injle  level  of  construction.  However,  because  of  the  larqe  amount  of 
Tirciitcy  required,  the  interconnection  problems  encouraqe  additional 
‘•ort  to  provide  more  compact  packaqinq. 

-rtitior  size  estimates  conclude  the  followinq: 


w 
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1.  The  SPJ1  rriqht  possitly  be  built  on  a single  minicomputer 
board  if  it  used  a simple  page  table,  limited  in  the  number 
of  pages  it  supported. 

2.  The  SP!1,  with  very  compact  packaging,  might  be  built  on  a 
single  board  if  it  used  a descriptor  arranged  associative 
system,  and  very  careful  attention  is  given  to  sharing  a 
memory  array  between  I/O  descriptor  support  and  processor 
descriptor  support. 

3.  It  is  safe  to  predict  tnat  the  SP:i,  with  associative 
descriptor  design,  could  be  built  on  two  boards. 

The  relative  cost  and  usefulness  of  different  designs  of  protection 
modules  have  been  estimated  by  outlining  trial  logical  designs,  and 
estimating  the  amount  of  equipment  it  would  take  to  realize  them. 
Highly  accurate  cost  estimates  were  not  expected;  good  relative  costs 
of  different  approaches  were  sought  to  guide  system  architectural 
decisions.  Three  basic  types  of  protection  units  were  considered. 


(a)  a simple  page  table,  with  the  simplest  possible  types  of 
protection  inaicatiors  added  to  each  fixed  size  page; 

(b)  augmentation  of  the  simple  page  table  to  obtain  a fairly 
elaborate  descriptor,  stored  in  a fast  access  memory.  A 
limit  field,  and  soire  space  reserved  for  software  usage, 
multiprocessor  controls,  and  other  unforeseeable  detail,  was 
added  to  create  a descriptor  of  four  words. 

(c)  augmentation  of  the  descriptor  to  one  reguiring  much  more 
elaborate  storage  space.  The  descriptor  was  expanded  to 
eight  words. 

The  relative  sizes  are  shown  in  Figure  19. 

It  was  found  that  there  was  an  irreducible  minimum  of  logic  and 
firmware  which  varied  relatively  little  from  the  minimum  to  the 
ma^xitruir  designs  considered.  Without  any  storage  elements,  the  simple 
page  table  scheme  was  estiirated  to  require  about  200  integrated 
circuits  (WSI) , while  the  support  for  the  S word  descriptor  requites  a 
minimum  of  322  RSI  circuits.  Tt  requires  about  hO  extra  IC's  to  get  a 
practical  unit  with  the  minimum  page  table  approach,  and  about  270 
extra  IC's  to  get  a practically  viable  unit  with  the  maximurr 
capability  8 word  descriptor.  The  most  elaborate  unit,  including  a 
general  purpose  cache,  requires  about  720  IC's. 
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Figure  19 


Relative  cost  and  usefulness  was  estimated  for  the  followinq 
alternative  desiqn  choices; 


♦ Seqment  size:  Fixed  iraximuir  lenqth  vs.  fixed  fraction  of  memory 

♦ t"omory  used;  Pull  descriptor  table  vs.  cache 

* Type  of  entry;  simple  paqa  table  vs.  seqment  descriptors 

* Kaximurr  size  descritabie  per  entry;  single  vs.  multiple 

The  use  of  a sinqle  fixed  size  page  table  per  process  with  fixed  size 
paqes  (similar  to  IDH  370)  provides  a much  simpler  hardware 
implementation  and  rapidity  of  access  to  the -page  descriptions  at  each 
use.  However,  this  simplicity  of  hardware  implementation  depends  upon 
choosing  the  full  oaqe  table  memory,  which  introduces  a substantial 
performance  penalty  each  time  the  context  of  the  processor's  usage  is 
shifted  from  one  process  (user  level)  to  another.  Designs  to  reduce 
this  penalty  depend  either  upon  a novel  miemory  technology,  or  upon  a 
larger  number  of  slower  integrated  circuit  memory  pacicages;  either 
approach  seems  technically  unwise.  Therefore  some  hardware  complexity 
should  be  accepted  and  the  choice  should  be  for  variable  size 
segments.  The  fastest  page  table  approach  also  requires  the  page  size 
to  remain  a constant  fraction  of  the  size  of  the  virtual  or  physical 
memory  managed  in  order  to  maintain  the  hardware  simplicity,  and  thus 
undesirably  constrains  all  software  designs.  Utilizing  the  constant 
descriptor  size  approach,  the  size  of  the  descriptor  table  memory  must 
increase  in  proportion  to  the  size  of  the  virtual  or  absolute  memory 
managed.  The  extra  circuit  delays  imply  performance  penalties  and 
physical  layout  complications,  as  well  as  higher  costs.  The  simple 
descriptor  table's  advantages  thus  appeared  chimerical. 

The  cache  memory  approach  risks  performance  loss  in  interrogating  the 
cache  and  then  fetching  data  into  the  cache  which  is  not  present  and 
must  be  extracted  from  memory.  This  also  implies  logic  and  firmware 
to  imolement  a replacement  algorithm.  However,  published  performance 
of  data  caches  indicated  that  "hit  ratios"  (probability  that  desired 
information  was  present)  exceeding  90%  could  be  expected  with 
practical  designs.  Multics  experiments  also  indicate  that  a very 
small  descriptor  cache  was  more  than  adeguate  for  obtaining  a high 
"hit  ratio."  See  Reference  20. 

The  fact  that  the  system  is  to  be  used  for  communicaticrs  makes  it 
desirable  to  define  independent  protection  for  relatively  small 
segments  of  memory  used  as  communications  buffers,  while  real  time 
procedure  characteristics  suggest  that  large  amounts  of  permanently 
resident  code  might  usefully  be  described  and  protected  by  a single 


descriptor.  If  only  larqe  iraximuin  size  seqments  can  be  described,  a 
sirall  entity  such  as  a communications  buffer,  a parameter  list,  or  a 
procedure  link  table,  can  te  independently  protected  only  by 
sacrificing  part  of  the  user's  virtual  space.  If  a small  maximum  is 
chosen,  more  memory  must  te  given  to  the  page  descriptor  tables,  and 
the  performance  is  reduced  when  page  tables  are  changed  in  context 
switching.  The  theoretically  optimum  size  for  a 512K  byte  memory 
(adequate  for  SATIN-IV)  is  rear  1024  bytes  - small  for  the  procedure 
and  large  for  the  buffers.  Furthermore,  if  the  fixed  page  size  are 
chosen,  physical  as  well  as  virtual  memory  would  te  wasted,  requiring 
either  more  time  lost  in  procedure  swapf inq  or  else  a larger  memory. 
This  favors  the  multiple  size  ability  in  alternative  4 above. 

The  above  interrelated  items  must  be  considered  together  before  any 
recommendation  can  be  chosen.  After  considering  the  impact  of  each 
item  uoon  the  others,  the  most  useful  design  would  use  a variable  size 
segment  description  ability,  with  a descriptor  cacne  realization,  and 
preferably  the  ability  to  use  at  least  two  qualitatively  different 
maximum  segment  sizes. 


TFADE-CFF  MOTES 


These  size  and  cost  considerations  result  in  a few  design  guiding 
tradeoff  rules.  The  most  useful  may  be  sum.marized: 

H^lJwaxe  Cost  vs  System  Growt]»  Potential 

1.  Increasing  virtual  address  support  by  1-4  bits  costs  only 
10-22  integrated  circuits  (IC's)  in  cache  like  descriptor 
storage,  while  storage  costs  double  for  each  additional  bit 
in  nor  cache  implementations-  Therefore,  increasing  virtual 
address  support  from  256  large  to  4096  small  pages  increase 
storage  cost  by  about  15  tim.es  in  a page  table 
implementation. 

2.  For  a 20-bit  virtual  address,  a simple  page  table  storage 
uses  the  least  circuitry;  for  a 24-bit  virtual  address  a 
cache  storage  uses  the  least  circuitry.  The  differences 
involved  a small  fraction  of  a minicomputer  board.  Moreover, 
at  24-bits  of  virtual  address  support,  the  medium  scale 
descriptor  unit  and  the  simple  page  table  have  the  same  cost; 
l<>.ss  than  two  minicomputer  boards.  This  suggests  that 
descriptor  ana  cache  techniques  are  not  inherently  too  costly 
or  complicated  to  use,  and  that  the  additional  system  growth 
potential  of  the  cache  comes  at  low  hardware  cost. 
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Size  ana  layout  Limits 

1.  There  seems  no  practical  way  to  achieve  the  design  goals 
without  at  least  one  minicomputer  hoard's  worth  of  circuitry. 

2.  It  is  difficult  to  visualize  a layout  of  the  interconnections 
needed  which  would  not  imply  that  the  protection  module  would 
grow  past  a single  hoard.  Physical  layout  permitting,  two 
adiacent  boards  seems  more  graceful. 

3.  The  probability  of  constructing  the  protecting  module  on  two 
boards  or  less,  with  current  technology,  seems  high,  assuming 
the  protected  processor  and  peripheral  units  could  provide 
the  necessary  information  for  control. 

P^IjfoiHance  Trade-off  ?Totes 

1.  Cache  usage  of  fast  access  storage  (100  - 200  nsec  access 
time)  is  slower  than  providing  a simple  page  table,  but  not 
but  more  than  one  logic  level  propagation  time  (about  10-15 
nsec)  per  use,  for  virtual  addresses  of  at  least  20  tits. 

• 2.  The  performance  penalty  for  increasing  the  size  of  the 

virtual  address  is  far  less  for  cache  than  for  page  table 
storage. 

3.  At  2h-tit  virtual  addresses,  there  appears  no  substantial 
difference  in  cache  vs.  page  table  performance. 

For  these  reasons  it  was  decided  that 

(a)  a cache  li)ce  unit  should  be  recommended  as  the  most  generally 

useful  design,  and 

(b)  it  should  strive  to  contain  a m.edium  scale  descriptor.  There 

is  only  a small  size  penalty  if  a larger  descriptor  were 
reguired. 


FLLIA3ILITY  CCFSIDEEATIONS 


This  section  addresses  Wree  considerations  related  to  the  reliability 
with  which  security  can  be  maintained  in  the  event  of  hardware 
failure.  They  are: 
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What  ate  useful  jrodels  of  the  relationships  between  hardware 
failure  and  security  breach? 


I 


2.  Hhat  practical  alqorithins  may  be  used  to  matce  realistically 
helpful  quantitative  predictions  of  the  security  of  a 
practical  system? 

3.  What  principles  should  be  followed  in  the  architecture,  • 
iirplerrentation,  and  confiqurat ion,  in  order  to  minimize  the 
risk  of  hardware  caused  security  breach? 

First,  let  us  consider  a model  for  estimating  limits  on  probability 
per  unit  tiire  of  security  breach  due  to  hardware  failure.  Gross, 
conservative,  order  of  magnitude  estimates  of  probability  of  security 
breach  may  be  made  by  assuming  that  any  hardware  failure  in  the  system 
leads  to  a potential  breach  of  security.  In  this  upper  limit 
estimating  technique  we  require  estimates  of  the  probability  per  unit 
time  of  1)  hardware  failure;  2)  undetected  error  due  to  hardware 
failure,  and  3)  security  breach  due  to  undetected  errcr. 

Probabilities  per  unit  time  are  usually  characterized  by  their 
reciprocals: 


1.  The  Kean  Time  between  Failure  (MTBF(i))  for  each  component 
(i)  of  the  system,  from  which  the  entire  system  MTBP  can  be 
derived  by 

1/HTBF  = 1/MTB?(1)  ♦ l/^TBP  (2)  *■  ...  1/?!TBF(n) 

2.  The  processing  activity  which  may  take  place  before  presence 
of  a failure  produces  a detected  error,  the  23ean  Processing 
Before  Error  detection  (MP3E) ; with  continuous  system 
ooeration,  the  Mean  Time  Before  Error  detection  (MTBl)  is 
equivalent. 

3.  The  probability  that  a given  fault  in  processing  results  in  a 
security  breach. 

The  MTEF  m.ay  be  estimated  for  most  physical  designs  using  current 
technology.  The  MTEF  is  a much  more  difficult  measure,  particularly 
in  the  presence  of  a possible  malicious  attempt  to  penetrate  the 
system.  In  fact,  no  complete  evaluation  can  possibly  be  made  until 
the  final  design  is  complete.  Adequate  design  guiding  estimates  of 
MTBE  due  to  hardware  failure  can  be  made  relative  to  the  MT3?  and  the 
system,  configuration  by  estimating  the  expectation  of  undetected 
) errors  oer  occurrence  of  a failure. 

1 1 


I 


It-  is  reasonable  to  assutre,  for  any  particular  iirplemer.tation,  that 
the  presence  of  a hardware  error  converts  a proper  infcriration  value 
to  an  incorrect  value,  with  probability  p(d)  that  it  is  recognizably 
incorrect.  The  inforiration  presented  on  successive  usages  of  the 
failed  natdware  may  often  be  assumed  sufficiently  uncorrelated  that 
successive  trials  may  be  regarded  as  nearly  statistically  independent. 
Ihen  the  expectation  of  undetected  errors  per  path  usage  car.  be  proven 
to  be  bounded  by  ^ 


(1  - P(d))/p(d)  . 

The  expectation  of  undetected  error  per  unit  processing  time,  1/MTBE, 
is  then  related  to  the  probability  of  failure  per  unit  processing 
time,  1/MTBF,  by 

1/MTEE  < (1  - F(d))/p(d) MTB?  (2qn.  6-1) 

where  in  practice  it  is  expected  that  (1  - p (d) ) <<  1/2,  so  that  the 
undetected  error  rate  is  substantially  less  than  the  failure  rate. 

This  provides  ar  upper  limit  estimate  of  the  probability  per  unit  time 
that  hardware  failure  causes  an  undetected  change  to  an  invalid 
security  state. 

The  probability  of  detecting  an  error  may  usefully  be  divided  into  a 
component  f (“r)  independent  of  the  .information  transmitted,  and  a 
component  o (r)  which  is  a function  of  the  information.  (For  example, 
single  bit  failures  in  a packet  of  information  checked  by  a parity 
circuit  are  always  detected;  multiple  bit  errors  ace  detected  with 
various  probabilities  depending  on  the  original  information.)  From 

p(d)  = f(“r)  ♦ (1  - f(“r))p(r),  , 

it  can  be  seen  that  p (d)  is  high  if  either  f (*r)  or  p (r)  is  high. 
Attention  to  practical  design  details  can  provide  both. 

For  example,  a high  fC^r)  may  be  found  in  transmitting  a single  byte 
and  parity  bit,  because  single  bit  errors  are  several  times  more 
common  than  multi-bit  errors.  Moreover,  power  failure  to  all  circuits 
transmitting  a 9 bit  parity  checked  byte  transmits  the  invalid  value 
03000000,0  or  the  value  11111111,1  - which  may  be  made  unlikely  or 
impossible  by  appropriate  system  design.  (Thus,  the  mest  significant 
byte  of  a virtual  address  may  be  required  to  be  other  than  11111111 
with  the  sacrifice  of  only  1/256  of  the  address  space.) 

Similarly  an  EDAC  memory  check  system  in  which  every  bit  transmitted 
participates  ir  more  than  one  parity  identity,  such  as  Hamming-related 
Single  Error  Correctirq  Double  Error  Detecting  (SEC-DED)  code, 
increases  p (r)  from  0.5  at  least  to  0.9375. 
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Inrplementinq  a set  of  paths  covered  by  a single  check,  such  as  the 
individual  bits  of  a byte  and  its  parity,  to  use  different  integrated 
circuits,  serviced  by  different  power,  clocking,  and  other  service 
leads  provides  even  greater  probability  of  early  detection  of 
failures.  For  s independent  circuits  involved  in  a checking  identity 
with  probability  p(d)  that  a failure  in  any  one  of  them  is  detectable, 

l/MTEP  = (1/KTBF)  (1-p  (d)  ) ♦*s/(1- (1-p  (d)  ) ♦♦s)  ) . (Fgn.  6-2) 

Since  (1  - F(d))  <<  1/2  in  practice,  even  a small  s such  as  2,  3,  or  4 
produces  a remarkable  lowering  of  the  undetected  error  rate  for  a 
aiven  p (d) , as  lable  III  shows. 

Practical  conditions  may  rule  out  universal  application  of  this 
principle,  but  it  may  be  specified  as  a design  goal  for  the 
implementor. 


TABLE  III 

P.eliability  Function  for  Independent  Fault  Detection 


P (d) 

X-(1-p(d)  ) 

s-1 

X**s/  (1 
s=2 

-X**s) 

s=3 

s=4 

0.6 

0.4 

0.667 

0.  1904 

0.0685 

0.0263 

0.75 

0.25 

0.333 

0.066 

0.016 

0.0039 

0.8 

0.2 

0.25 

0. 0416 

0.008 

0.0016 

0.9 

0.1 

0.11 

0.0101 

0.001 

0.0001 

BouDi,  Prohahilitv  of  Breach  of  Security 

We  can  estimate  maximum  probability  of  breach  of  security  as  a 
function  of  expected  failure  rate,  and  details  of  system 
implementation.  Numerical  values  of  an  upper  bound  on  the  probability 
of  security  breach  per  unit  time  may  be  estimated  by 

1/MTBB  = PS/(1  ♦ P)  (1  ♦ S) MTBE  (Eqn.  6-3) 

where  S is  the  ratio  of  system  time  spent  in  manipulating  security 
related  information  to  system  time  spent  in  processor  and  memory 
manipulations  in  an  equivalent  unprotected  computer  system,  and  P is 
the  ratio  of  the  amount  of  logic  and  supporting  equipment  in  the 
protection  subsystem  to  that  in  the  unprotected  system.  The  ratio 
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(S/(1  ♦ S))  represents  the  probability  that  a hardware  fault  first 
affects  security  related  inforiration,  rather  than  data  or  procedure. 
Presumably,  usirq  either  gives  the  hardware  error  detecting  redundancy 
an  equal  opportunity  to  indicate  the  presence  of  trouble  which  will 
cause  the  system  to  be  stopped,  or  reconfiqured  and  deflected  to  test 
and  diaqnostic  actions. 

Estimate  of  the  HTB'P  p(d)  factor  for  various  integrated  circuits, 
cornectiors,  and  subsystems  tray  be  done  by  statistically  valid 
simulated  operation  of  each  subsysteir  with  simulated  introduction  of  a 
(randorrly  selected)  permanent  failure  in  the  hardware.  It  could  also 
he  done  by  analysis  of  performance  of  similar  circuits  with  standard 
•fault  analysis'  algorithms  well  known  in  the  literature.  (11) 

? better  method,  usable  only  when  a logic  design  is  complete, 
estim.ates  probability  of  security  breach  by  full  system  logic  analysis 
and  simulation  (12)  methods  used  tor  generating  hardware  logic 
diaqnostic  tests.  An  efficient  and  thorough  set  of  such  tests  may  be 
used  as  a base  for  the  generation  of  others  which  will  test  the  system 
more  thoroughly  than  any  practical  detailed  test  design. 

The  best  method  is  provision  of  appropriate  redundancy  checks,  and 
extensive  subsystem  MIBB  analysis  during  the  design  phase.  Expected 
ranges  of  !1T3F  for  practical  systems  in  the  minicomputer  range,  using 
modern  technologies,  should  be  in  the  thousands  of  hours,  and  that  for 
the  subsystems  cf  individual  logic  paths,  germane  in  the  above 
context,  should  be  in  the  tens  or  hundreds  of  thousands  of  hours.  It 
is  best  to  design  as  many  subsystems  as  possible  to  provide  an 
estimated  MTEE  which  satisfies  MAXUSEMTBE  in 

HAXOSEHTBE  > (Ns)  (LIFE)  / (PIS K)  , where 
Ns  = the  number  of  systems  expected  to  be  in  use,  and 
LIFE  = the  expected  life  of  a system,  and 

BISK  = the  risk  it  is  reasonable  to  assume  acceptable  for  a 

security  breach  on  any  one  of  the  Ns  systems  during  their 
installed  life. 

As  an  example,  consider  a practical  system  of  twelve  beards  of 
minicomputer  electronics  and  an  MTEF  of  8000  hours.  Adding  to  this 


(11)  c.f.  Symposia  on  Fault-Tolerant  Computing,  PTC-1  to  FTC-5 
inclusive,  published  1971  to  1975  by  IEEE,  New  York. 

(12)  ibid. 
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an  SPM  of  approximately  1-1/2  toards  of  logic  changes  the  MTBF  to 
MTBr  * r 12/(12  + 1.5)  ]f  8000]  * 7100  hours 
Then  Equation  6-1  iray  be  used  to  estimate  that  MTBE  exceeds 

MTE£  > 7100  fO. 9/0.1]  = 63,900  hours 
for  the  entire  system. 

In  operation,  the  system  carries,  security  related  (absolute) 
addressing  information  on  the  bus  about  305  of  the  time  any  bus  is  in 
use.  Thus  (3/ ( 1 + S)  ) =0. 3 and  F/(1+P)  = (1. 5/(13. 5))  = 1/9,  and 

nTBB-S  9/0.3(MTEE)  = 30(63,  900)  * 1,920,000  hours. 

This  technique  must  be  applied  to  the  designed  system  for  an  adequate 
comoination  of  subsystems,  using  median,  minimum,  and  maximum 
estimates  for  KTBr,  p (d) , and  the  ratio  S?/(1+S)  (1  + P) . The  resulting 
distribution  of  estimates  gives  a measure  of  the  confidence.  For 
qreater  assurance,  a reasonable  distribution  may  be  assigned  each 
subsystem's  estimate,  and  MonteCarlo  calculation  of  the  entire 
system's  JTBB  estimate  made. 

Table  IV  outlines  a minimum  set  of  analyses  required  for  a practical 
system.  Where  the  MIBB  estimate  is  too  small,  checking  circuitry  must 
be  introduced  into  the  design.  The  redundancy  is  the  only  way  to 
increase  p (d)  so  that  the  system  MTEB  may  be  adequate.  There  will  be 
certain  circuits  critical  to  security  for  which  this  necessity  is  to 
be  expected. 

.sr  space  3 


TAELE  IV 


Siibsvstetrs  in  Analysis 


Security-Eelated  Processing 


1 • Outside  5 PK 

Address  bus  5 checks 
and  memory  module 
address  circuitry. 

Address  control 


Data  bus  & checks 


Data  Control 

Other  bus  controls 

Interrupt  network 
priority  resolver 


Absolute  only,  iff  new  address  out 
of  user’s  space 


Absolute  only,  iff  SA1  failure, 
(time  out  if  SAC) 

Only  when  passing  descriptor  parts, 
critical  state  informaticn,  or 
absolute  device  identification,  and 
classes  of  errors  as  on  A-bus 

SA1  failure,  either  A-bus  or  V-bus 

To  be  analyzed 

Only  for  SPM  security  fault  con- 
dition being  transformed  to 
other  fault  coi»dition 


Timing  information 


Power  and  ground 

2 . Inside  SP H 

Associator  identifying 
descriptor 

Descriptor  permission 
information;  permission  permission  ’ 

checking  logic  and  storage 

Address  within  descriptor  Iff  altered  base  is  out  of  user's 

space 

Limit  within  descriptor  Iff  limit  be  effectively  increased, 

and  overlaps  another  user's  resource. 


If  withheld  from  kernel,  or  if 
"unique-name”  generated  from  clock 
be  repeated. 


To  be  analyzed 


( 


on 


Cnly  for  false  "hit"  indicati| 
Only  for  false  extension  of  \ 
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I 


{i 


Current  user_id.  Always  potential  breach 

operating  ring 


DESIGN  RECCMMENEATIONS  - RELIABILITY 


To  iriniirize  the  probability  of  security  breach  per  unit  time,  it  is 
recorrmenied  that  the  systerr  itrplerrentation  both  of  hardware  and 
software  malte  use  of  the  followinq  principles: 

1.  The  hardware  should  irake  extensiye  use  of  redundancy  codinq 
wherever  practical-  All  merrories  and  busses  should  use 
parity  and/or  Sinqle  Error  Correcting,  Double  Error  Detecting 
Hairirinq  code,  or  the  equivalent,  to  increase  the  MTBF.  This 
particularly  includes  parity  in  the  descriptor  cache,  and  on 
address  and  data  busses. 


To  keep  f (“r)  an 
is  security  crit 
irrplemented  to  u 
supportable  by  t 
smaller  size  of 
practical  system 
as  "always  zerc" 
software  should 
similar  prohafail 
address  space,  r 
quantity  for  all 
of  the  virtual  a 
f (“r)  is  also  a 
breach  probabili 


d p(r)  high  in  the  address  circuitry,  which 
ical,  hardware  and  software  should  be 
so  the  maximum  internal  virtual  address  space 
he  internal  physical  address  bus,  even  if  the 
the  physical  memory  a'ssociated  with  a given 
makes  it  tempting  to  wire  some  address  leads 
both  in  hardware  and  in  software.  Note  that 
assign  virtual  internal  addresses  with 
ity  throughout  the  physically  supportable 
ather  than  starting  at  the  same  small 
allocations.  Sacrificing  a small  fraction 
ddress  potential  for  the  sake  of  increased 
valuable  opportunity  to  decrease  security 
ty- 


The  MTEP  of  the  security  critical  circuitry  and  firmware 
should  be  made  very  high.  If  the  implementor’s  estimated 
WTBF  for  a critical  subsystem  does  not  exceed  desired  MTBB, 
he  must  either 

(a)  provide  redundant  logic,  raising  p(d)  to  detect 
inconsistency  in  time  to  prevent  damage  to  the  system,  or 
else 

(b)  describe  the  frequency  f of  diagnostic  error  checks 
which  must  be  introduced  during  system  operation  to 
quaraiitee  that 


1/f(MT3r’)  < risk 
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for  these  critical  parts  of  the  system,  for  the  desired 
maximum  risk  of  hardware  caused  security  breach  between 
successful  operations  of  the  diagnostic  checks. 


PZB?CRMANC5  ESIIKATES 


Tn  the  foreseeable  state  of  the  art,  there  appears  no  way  to  implement 
a secure  system  without  some  performance  penalties.  Some  mention  has 
been  made  of  the  implementation  effects  on  perform.ance,  and  some  of 
our  design  recommendations  have  been  made  with  the  intent  of  avoiding 
severe  performance  penalties.  This  section  discusses  the  size  of  the 
performance  penalties  which  may  practically  be  expected  for  a system 
using  an  SPM,  compared  with  an  ordinary  unsecured  system.  It  is 
assumed  that  the  SPtI  is  to  be  added  to  a system  of  approximately  the 
following  characteristics; 

♦ The  processor  meets  the  requirements  listed  earlier. 

♦ The  main  memory  cycle  is  1 usee  +250  nsec. 

♦ The  unprotected  system  has  no  data  or  instruction  cache  not 
controllable  by  the  SEM.  (The  S?M*s  must  manage  all  cache's.) 

♦ The  system  is  prepared  for  multiprocessor  installation, 

♦ Each  processor  of  the  system,  except  those  connected  to  appear 
as  peripheral  devices,  has  its  own  SPM. 

♦ All  peripheral  controllers  are  connected  to  the  virtual 
(•^uncertified”)  bus. 

♦ There  is  no  buffering  of  data  or  addresses  within  the  SPM, 
except  for  descriptor  working  registers,  and  the  descriptor 
cache  and  peripheral  descriptor  storage. 

♦ direct  Memory  Accesses  (DMA)  required  by  peripheral  data 

transfers  comprise  less  than  10%  of  the  total  memory  accesses. 
(This  is  suggested  by  extensive  metering  and  simulations  of 
many  systems.) 


I * Context  switching  is  assumed  to  occur  as  often  as  once  per 

‘ 1000  to  2000  memory  references,  based  on  Schiller’s  MITF.E 

PDP/11/45  security  kernel  design  study,  as  an  upper  limit. 
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It. is  understood  that  tho  reference  monitor  software  may  be  desiqned 
to  be  resident  in  every  user’s  space  at  all  times,  requirinq  less 
effort  than  a full  context  switch  for  activation.  These 
considerations  will  be  iqnored  at  first  in  favor  of  obtaining  a 
qrosslv  conservative  estimate  of  performance  loss. 

Sources  of  Jeduced  Ferf ormance 

Three  principal  causes  may  be  discerned  for  inherently  reduced 
performance  due  to  presence  of  a Security  Protection  Module.  They 
are: 

* the  time  required  to  generate  and  manipulate  descriptors; 

* the  time  required  to  place  a descriptor  in  cache  to  use  it; 

* the  additional  delay  in  each  reference  to  memory  due  to 
descriptor  cache. 

To  these  must  be  added  and  subtracted  the  following  components  whose 
presence  and  extent  are  implementation  dependent; 

♦ the  extra  "shew''  or  "synch”  delay  in  some  two  bus 
implementations,  with  certain  types  of  logic  and  memory 
circuitry  in  the  SFM;  this  is  also  associated  with 

* the  time  lost  by  distributing  peripherals  between  two  busses 
rather  than  one.  The  bus  time  is  usually  less  than  a linear 
function  of  the  number  of  peripherals  connected; 

♦ the  duplicated  parts  of  a bus  cycle,  under  the  same 
circumstances ; 

♦ the  time  lost  by  descriptors  contending  for  places  in  the 
descriptor  cache.  This  factor  is  cache  size  and  algorithm 
dependent;  sound  design  should  keep  it  to  a fraction  of  a 
percent. 

♦ the  time  saved  if  the  necessary  request  storage  is  implemented 
to  serve  also  as  a data  cache; 

If  multiole  central  processors  are  present,  there  is  an  additional 
increment  to  each  of  thase.  To  a first  approximation,  it  will  be 
assumed  this  is  in  the  same  proportion  for  protected  and  unprotected 
systems,  except  as  expressly  indicated  otherwise  in  the  analysis. 

£££iSXiran£e  fSdufiJLcQ  S^ijrss  1 
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The  tiire  required  to  Generate  and  manipulate  descriptors  is  an 
acplication  dependent  source  of  performance  reduction  believed 
substantially  less  than  either  of  the  others.  Descriptors  must  be 
qenerated  or  manipulated  only  when  a physical  resource  must  be 
temporarily  reassiqned  {paqe  in  or  page  out)  or  when  a process'  or 
use(r)*s  work  is  initiated  or  terminated.  Most  of  the  work  is  memory 
manaqement  which  must  be  performed  in  some  manner.  In  communications 
particularly,  the  real  time  constraints  frequently  require  much  of  the 
procedure  to  be  resident  in  memory  at  all  times,  further  reducing  the 
demand  for  descriptor  qeneration  or  alteration.  No  adequate 
quantitative  model  has  been  m.ade  of  this  performance  increment,  to  the 
writer's  knowledqe.  The  presence  of  descriptors  and  their 
manipulations  may  even  be  a plus,  by  providing  convenient  discipline 
and  hardware  assistance  for  these  necessary  manipulations. 

P^formance  Eeduction  Source  2 

A process  residency  will  be  defined  as  the  processing  between  events  - 
such  as  a wait  for  I/O  transfer,  or  assignment  of  a new  peripheral 
resource  - which  cause  enough  delay  to  make  dispatch  of  a new  process 
highly  probable.  Assume  three  words  of  a descriptor  must  be  placed  in 
the  cache  before  referencing  can  be  done,  although  a fixed  page  size 
subdescriptor  type  might  be  defined  requiring  only  two  words  of 
transmission.  Kith  a Head-Alter-Write  memory,  setting  the  0 (sed) , 

M (odified) , and  C (encacheable)  bits  takes  no  extra  time,  except  for 
write  references  made  on  other  than  the  first  use  of  a descriptor; 
these  require  ar  extra  cycle. 

Let  p (DW)  represent  the  fraction  of  the  descriptors  first  referenced 
for  writing  after  their  first  use,  and  k represent  the  control  time  to 
initiate  a descriptor  fetch.  The  lost  descriptor  loading  time  in  a 
process  residency  referencing  n descriptors  is 

(3  ♦ k ♦ p(DH))n 

memory  cycles. 

In  an  m-processor  system,  some  descriptor  modifications  must  be  noted 
by  other  use(r)s  of  that  descriptor.  They  will  receive  a message  from 
the  Clear  Associative  Memories  (CAM)  function  to  invalidate  all  copies 
of  any  descriptor  "SIMILAP”  to  the  one  modified  - by  a conveniently 
built  criterion  of  SIMiLAPity.  The  time  lost  is  not  only  that  assum^ed 
necessary  to  create  the  descriptor,  but  also  the  time  spent  refetching 
**SIHILAR''  descriptors  in  other  processors. 

If  p (D)  rooresents  the  probability  that  a descriptor  has  been  deleted 
from  cache  between  residencies,  and  E(S)  is  the  expectation  per 
process  of  encached  descriptors  which  match  the  SIMILAEity  criterion 
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of.  CAM,  th-9  extra  tiire  is  approximately 
j p(D)n'E(S)  (3  ♦ k ♦ p(DW)) 

iremory  cycles. 

The  definition  of  the  symbols  p(D)  and  E(S)  is  a great  deal  easier 
than  their  calculation,  which  is  heavily  a function  of  the  details  of 
the  SIMIlA^ity  algorithm,  the  number  and  nature  of  processes  competing 
for  space,  etc-  The  confidence  that  useful  operation  exists  is  again 
provided  by  Multics  experience.  The  proposed  system  should  expect  to 
operate  with  fewer  page  displacements  oer  machine  language  program: 
step  than  Multics. 

•I 

1 There  is  reason  to  suspect  that  the  product  p(D)E(S)  <*  1 with  many 

' likely  SIMILAP.ity  functions  and  with  some  reasonable  assumptions  for 

the  order  of  magnitude  of  p(D).  Figure  20  graphs  this  estimated 
I performance  reduction  component. 

j Figure  20  also  provides  a rough  estimate  of  the  cost  of  increasing  the 

expected  number  of  descriptors  used  per  thousand  memory  references. 

For  estimating  convenience,  the  "extra  cycles”  scale  is  ten  times  the 
percent  performance  loss  in  memory  limited  system  operation,  if  there 
arc  one  thousand  references  per  residency. 

In  practice  the  number  of  descriptors  used  per  1000  memory  references 
will  tend  to  remain  less  than  12- 1h,  without  tremendous  care  in 
program  design.  (13)  A low  number  of  descriptors  referenced  per  usage 
is  particularly  important  in  designing  the  freguently  used  reference 
mcritor- 


Ferformance  Peduct ion  Source  3 


Delay  in  the  descriptor  evaluating  and  permission  circuit  checking  was 
estimated  bv  a trial  logical  design,  using  typical  currently  available 


(13)  c.f.  Fef.  20  - Schroeder,  M.D.  "Performance  of  the  GE-645 
associative  memory  while  Multics  is  in  operation”,  for  Multics 
experience.  Practical  studies  in  design  of  "based"  computer  systems 
such  as  the  IBM  360/370,  Honeywell  62  and  2000  series,  etc.  have  also 
indicated  that  fi-16  bases  are  very  convenient  for  defining 
I addressability  needed  in  the  locality  of  a program,  with  less  than 

5 five  percent  of  the  program’s  time  spent  managing  its  base  registers. 
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components 


Nature  of  Desiar 


Estimated  Delay 


Hiqhly  optimistic  75  nsec 

Realistic  125  nsec 

Reasonable  Goal  100  nsec 

Fioure  21  displays  the  percent  of  time  added  from  this  source,  as  a 
function  of  the  ireirory  module’s  peak  access  rate  (horizontal  scale) 
and  the  percentage  of  system  operation  which  is  memory  access  limited 
(choice  of  three  vertical  scales).  For  memory  cycle  times  in  the 
range  of  lOOOnsec,  and  a typical  60^  to  80%  of  system  time  limited  by 
mem.ory  accesses,  the  memory  performance  loss  due  to  this  source  could 
be  exoected  to  be  less  than  105. 

Performance  Changes  - JUIllementation  - Dependent  Components 


The  separation  of  the  peripherals  into  two  different  groups  in  a two 
bus  implementation  is  likely  to  imply  that  the  resynchinq  time 
required  on  a parallel  bus  is  experienced  twice  - once  on  the  virtual 
bus,  and  once  on  the  absolute  bus.  This  may  not  be  true  if  the 
associative  logic  can  be  cascaded  without  resynchronization,  and/or  if 
special  anticipative  logic  interfaces  can  be  provided  for  all  bus 
users,  but  this  is  unlikely  to  be  practical.  It  is  likely  to  be  true 
if  the  memory  does  require  a resynchronization  at  its  addressing 
input.  This  component  of  delay  might  typically  be  an  extra  25  nsec 
with  present  technoloq?.es,  representing  2.5%  extra  system  time  with  a 
1 usee  memory  and  a 100%  memory  limited  operation. 

If  resynchronizat ion  is  required,  the  SPH  cannot  begin  its  action 
until  almost  an  entire  virtual  bus  cycle  has  been  completed.  It  must 
wait  for  the  last  element  of  skew  delay  on  the- V-Bus  before  it  can 
either  (a)  initiate  the  actions  on  the  V-Bus  to  acknowledge  the 
information  and  release  the  bus  for  other  usage,  or  (b)  initiate 
consultation  of  the  descriptor  memory. 

The  situation  is  much  worse  if  the  bus  response  time  also  must  be 
duplicated  on  both  busses.  This  introduces  a non-trivial  component  of 
♦he  order  of  100  to  150  nsec.  Fortunately,  a bus  system  organization 
requiring  bus  response  duplication  may  well  also  imply  the  graceful 
opportunity  to  use  the  necessary  request  storage  as  data  and 
instruction  cache.  Peferences  to  this  extra  general  purpose  cache  may 
save  a great  part  of  the  performance  otherwise  lost. 
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SUMMARY  - EXPECTED  PEPFCRMANCE 


The  performance  deqradation  caused  by  the  SPM  may  be  estimated  by 
addinq  expected  components  of  deqradation. 

For  example,  assume  a system  with  a 1000  nsec  memory  cycle,  a 100 
nsec  descriptor  cache  delay,  approximately  75X  of  its  time  spent 
memory  limited,  and  not  more  than  two  multiprocessors.  Assume  also 
that  about  10  different  descriptors  are  expected  to  be  referenced 
per  processor  residency  of  1000  memory  cycles  between  context 
switchinq.  The  total  system  deqradation,  compared  to  an  equivalent 
unsecured  system,  is  of  the  order  of  17%. 

Experimentation  indicates  it  is  very  difficult  to  set  up  circumstances 
which  would  lead  to  more  than  25S  performance  degradation  with  likely 
assumptions  for  the  "Implementation  dependent"  components.  More 
precise  estimates  require  implementation  details,  especially  the 
amount  of  overlap  between  SPM  and  processor  operations,  and  the 
relationships  of  the  m.ost  used  proqrams'  working  set  reference 
patterns  to  the  organization  of  the  caches  in  the  SPM. 


PEPFCRMANCE  ANALYSIS  - STJKMARY 


These  notes  provide  estimating  guides,  which  serve  to: 

* Indicate  to  programmers  useful  trade-offs  in  using  many  versus 
few  descriptors  in  their  program  organizations. 

* Indicate  to  implementors  useful  trade-offs  in  reducing  the 
effective  delay  by  clever  design  - a challenge  when  the 
designer  of  the  protected  miniprocessor  is  already  attempting 
to  push  technology  to  its  utmost. 

* Establish  the  need  for  further  analysis  to  describe  the  best 
design  trade-offs  in  multiprocessor  systems. 

• * Provide  practical  indications  that  the  performance  cost  of 

protection  is  economically  acceptable. 
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APPENDIX  A 


Because  of  funding  limitations,  the  Air  Force  terminated  the  effort  j 
which  this  document  describes  before  the  effort  reached  its  logi-  i 

cal  conclusion.  The  Air  Force  comments  which  were  present  at  the  I 

time  the  effort  was  terminated  are  as  follows:  j 

, 1.  Page  9,  line  11  - Multics  was  selected  because  it  provided  | 

the  best  hardware  and  its  software  was  the  most  adaptable  ] 

and  not  because  it  was  "least  vulnerable-"  i 

2.  Page  13,  line  1 - How  were  the  criteria  for  a~ "properly 
organized"  segmented  memory  determined? 

3.  Page  15,  line  7 from  bottom  - The  comparison  of  the  SPM 

k and  the  ideal  reference  monitor  based  on  the  figures  is 

a gross  oversimplification.  The  SPM  functions  are  more 
important  than  its  interfaces  to  other  elements  in  the 
system. 

4.  Page  21,  line  16  - The  last  sentence  of  the  paragraph 
presents  a faulty  dilemma.  One  multilevel  secure  system 
could  replace  several  single  level  systems  and  still  not 
provide  an  information  sharing  capability. 

5.  Page  33,  line  4 - The  point  of  this  paragraph  is  not 
clear.  Function  x should  not  be  in  kernel  if  it  was  not 
security  related.  The  example  illustrates  a case  where 
progreuns  for  one  level  of  abstraction  may  reside  in  more 
than  one  domain  and  is  not  necessarily  a conflict  as 
claimed.  The  ring  mechanism  is  not  the  only  isolation 
mechanism.  Processes  can  also  isolate. 

6.  Pages  36  to  44,  General  - 

a.  Many  unexplained  design  decisions  are  found  on  these 
pages . 

b.  The  discussion  of  software  levels  was  confusing. 

I The  reader  may  have  difficulty  at  each  level 

^ distinguishing  what  was  being  implemented  from  what 

was  available  for  the  implementation. 
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12. 


7. 


8. 


9. 


10. 


11. 


Page  50,  line  4 from  bottom  - The  definition  of  pro- 
cesses differs^ from  the  one  on  page  42.  

Page  53,  and  following  pages  - More  unexplained  design 
decisions  are  found  here. 

Page  65,  line  16  from  bottom  - The  Multics  clock  which 
records  time  from  1 Jan  1900  to  late  in  the  year  2042  is 
a red  herring. 


Page  66  and  following  - 
appear  here. 


More  unexplained  design  choices 


Page  72  and  following  - Many  realization  requirements 
are  unsupported  (nine  bit  byte  potential) , questionable 
(the  SPM  could  provide  a real  time  clock  as  easily  as 
the  base  minicomputer)  or  wrong  (the  PDP  11/45  could  have 
an  SPM  but  has  few  reserved  operation  codes) . 

Page  55  and  following  - Many  of  the  implementation 
considerations  seem  to  assume  a 16  bit  minicomputer  or 
even  a HIS  Level  6. 
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